A family of stochastic processes whose realizations are probability distributions
From "Dirichlet Process" by Yee Whye Teh:
The Dirichlet process is a stochastic proces used in Bayesian nonparametric models of data, particularly in Dirichlet process mixture models (also known as infinite mixture models). It is a distribution over distributions, i.e. each draw from a Dirichlet process is itself a distribution. It is called a Dirichlet process because it has Dirichlet distributed finite dimensional marginal distributions, just as the Gaussian process, another popular stochastic process used for Bayesian nonparametric regression, has Gaussian distributed finite dimensional marginal distributions. Distributions drawn from a Dirichlet process are discrete, but cannot be described using a finite number of parameters, thus the classification as a nonparametric model.
From Wikipedia:
Dirichlet processes are usually used when modeling data that tends to repeat previous values in a "rich get richer" fashion. Specifically, suppose that the generation of values $X_{{1}},X_{{2}},\dots$ can be simulated by the following algorithm.
Input: $H$ (a probability distribution called base distribution), $\alpha$ (a positive real number called concentration parameter) Draw $X_{{1}} $ from the distribution $H$. For $n>1$:
- With probability ${\frac {\alpha }{\alpha +n-1}}$ draw $X_{{n}}$ from $H$.
- With probability ${\frac {n_{{x}}}{\alpha +n-1}}$ set $X_{{n}}=x$, where $n_{{x}}$ is the number of previous observations $X_{{j}},j \lt > n$, such that $X_{{j}}=x$.
At the same time, another common model for data is that the observations $X_{{1}},X_{{2}},\dots$ are assumed to be independent and identically distributed (i.i.d.) according to some distribution $P$. The goal in introducing Dirichlet processes is to be able to describe the procedure outlined above in this i.i.d. model.
