Why the sample method of mixture distribution works?

Question

For example this thread: Generating random variables from a mixture of Normal distributions

1. First choose a distribution according to the weights.
2. Then sample from the chosen distribution.

How to prove the correctness of this method?

Answer 1

Here's how I think about it.

Let \begin{equation} p(y,z) = p(y|z)\,p(z) , \end{equation} where \begin{equation} p(y|z) = \textsf{N}(y|\mu_z,\sigma_z^2) \end{equation} and \begin{equation} p(z = c) = \begin{cases} w_c & c \in \{1, \ldots, K\} \\ 0 & \text{otherwise} \end{cases} \end{equation} and $w_c \ge 0$ and $\sum_{c=1}^K w_c = 1$. We can make a draw from the joint distribution by first drawing $z$ from its marginal distribution and then drawing $y$ from its conditional distribution.

The marginal distribution for $y$ is given by integrating out $z$. Because $z$ is discrete, this amounts to \begin{equation} p(y) = \sum_{c=1}^K p(y|z)\,p(z=c) = \sum_{c=1}^K w_c\,\textsf{N}(y|\mu_c,\sigma_c^2) . \end{equation} Thus, the marginal distribution for $y$ is a mixture. We may draw from this marginal distribution by drawing $z$ and $y$ from the joint distribution and discarding $z$.