3

I would like to know the difference between a multivariate Gaussian distribution and multivariate Gaussian mixture model. Can someone provide an intuitive and/or detailed explanation?

Thanks.

notArefill
  • 279
  • 2
  • 4
  • 13
  • See, for example, [this question](http://math.stackexchange.com/q/195911/15941) for a multivariate Gaussian mixture model. – Dilip Sarwate Sep 03 '15 at 21:30

1 Answers1

3

A (non-degenerate) multivariate Gaussian density function has a specific form $$f(\mathbf x) = \frac{1}{(2\pi)^{n/2}\sqrt{\det(\Sigma)}} \exp\left(-\frac{1}{2}\left(\mathbf x - \mathbf m\right)\Sigma^{-1} (\mathbf x - \mathbf m)^T \right).\tag{1}$$

A multivariate Gaussian mixture model is a weighted sum of densities such as $(1)$:

$$f(\mathbf x) = \sum_{i=1}^N p_i\cdot\frac{1}{(2\pi)^{n/2}\sqrt{\det(\Sigma_i)}} \exp\left(-\frac{1}{2}\left(\mathbf x - \mathbf m_i\right)\Sigma^{-1}_i (\mathbf x - \mathbf m_i)^T \right)$$ where the $p_i$ are positive numbers such that $\sum_{i=1}^N p_i = 1$. Effectively, we have $N$ mutually exclusive events $A_i$ such that the conditional density given that $A_i$ occurred is $$\frac{1}{(2\pi)^{n/2}\sqrt{\det(\Sigma_i)}} \exp\left(-\frac{1}{2}\left(\mathbf x - \mathbf m_i\right)\Sigma^{-1}_i (\mathbf x - \mathbf m_i)^T \right)$$ and $(2)$ is just the law-of-total-probability expression for the unconditional density.

Dilip Sarwate
  • 41,202
  • 4
  • 94
  • 200
  • I would appreciate it if you can have a look at my other question http://stats.stackexchange.com/questions/171307/how-to-train-a-gaussian-mixture-hidden-markov-model – notArefill Sep 06 '15 at 10:38
  • This can be done completely generally by using distribution or characteristic functions instead of densities. That allows for one or more of the Gaussians being mixed to be degenerate. – Mark L. Stone Feb 18 '17 at 23:40