A Bayesian nonparametric model is a Bayesian model on an infinite-dimensional

parameter space. The parameter space is typically chosen as the set of all possible

solutions for a given learning problem. For example, in a regression problem

the parameter space can be the set of continuous functions, and in a density estimation

problem the space can consist of all densities. A Bayesian nonparametric

model uses only a finite subset of the available parameter dimensions to explain

a finite sample of observations, with the set of dimensions chosen depending on

the sample, such that the e ffective complexity of the model (as measured by the

number of dimensions used) adapts to the data. Classical adaptive problems,

such as nonparametric estimation and model selection, can thus be formulated

as Bayesian inference problems. Popular examples of Bayesian nonparametric

models include Gaussian process regression, in which the correlation structure

is refined with growing sample size, and Dirichlet process mixture models for

clustering, which adapt the number of clusters to the complexity of the data.

Bayesian nonparametric models have recently been applied to a variety of machine

learning problems, including regression, classification, clustering, latent

variable modeling, sequential modeling, image segmentation, source separation

and grammar induction.

In this reading group we will be going over Michael I. Jordan’s tutorial presentation from NIPS focusing on Dirichlet processes.

The original post script file can found in the following link:

http://www.cs.berkeley.edu/~jordan/nips-tutorial05.ps

The presentation follows the same lines as the paper found in:

http://www.cs.berkeley.edu/~jordan/papers/hdp.pdf