A Bayesian nonparametric model is a Bayesian model on an in finite-dimensional
parameter space. The parameter space is typically chosen as the set of all possible
solutions for a given learning problem. For example, in a regression problem
the parameter space can be the set of continuous functions, and in a density estimation
problem the space can consist of all densities. A Bayesian nonparametric
model uses only a fi nite subset of the available parameter dimensions to explain
a finite sample of observations, with the set of dimensions chosen depending on
the sample, such that the e ffective complexity of the model (as measured by the
number of dimensions used) adapts to the data. Classical adaptive problems,
such as nonparametric estimation and model selection, can thus be formulated
as Bayesian inference problems. Popular examples of Bayesian nonparametric
models include Gaussian process regression, in which the correlation structure
is re fined with growing sample size, and Dirichlet process mixture models for
clustering, which adapt the number of clusters to the complexity of the data.
Bayesian nonparametric models have recently been applied to a variety of machine
learning problems, including regression, classifi cation, clustering, latent
variable modeling, sequential modeling, image segmentation, source separation
and grammar induction.
In this reading group we will be going over Michael I. Jordan’s tutorial presentation from NIPS focusing on Dirichlet processes.
The original post script file can found in the following link:
http://www.cs.berkeley.edu/~jordan/nips-tutorial05.ps
The presentation follows the same lines as the paper found in:
http://www.cs.berkeley.edu/~jordan/papers/hdp.pdf