Drawing from Dirichlet distribution

Question

Let's say we have a Dirichlet distribution with $K$-dimensional vector parameter $\vec\alpha = [\alpha_1, \alpha_2,...,\alpha_K]$. How can I draw a sample (a $K$-dimensional vector) from this distribution? I need a (possibly) simple explanation.

Answer 1

First, draw $K$ independent random samples $y_1, \ldots, y_K$ from Gamma distributions each with density

$$ \textrm{Gamma}(\alpha_i, 1) = \frac{y_i^{\alpha_i-1} \; e^{-y_i}}{\Gamma (\alpha_i)},$$

and then set

$$x_i = \frac{y_i}{\sum_{j=1}^K y_j}. $$

Now, $x_1,...,x_K$ will follow a Dirichlet distribution

The Wikipedia page on the Dirichlet distribution tells you exactly how to sample from the Dirichlet distribution.

Also, in the R library MCMCpack there is a function for sampling random variables from the Dirichlet distribution.

Answer 2

A simple method (while not exact) consists in using the fact that drawing a Dirichlet distribution is equivalent to the Polya's urn experiment. (Drawing from a set of colored balls and each time you draw a ball, you put it back in the urn with a second ball of the same color)

Consider your Dirichlet parameters $\alpha_i$ as an unormalized distribution over i.

Then :

repeat N times

--> draw an i using the $\alpha_i$ multinomial distribution

--> add 1 to $\alpha_i$

end repeat

Normalize $\alpha$ to get your distribution

If I am not wrong, that method is asymptotically exact. But since N is finite, you will NEVER draw some distributions with very small prior probabilities (while you should draw them with a very small frequency). I guess it might be satisfying in most cases with N = K.10.