1

I just would like to understand some information about the joint normality and the margins. I read that the normal joint distribution almost always implies that the univariate margins are all normal.

My question is:

1- Is this apply for Gaussian mixture models? For example, in the Gaussian mixture model, each mixture component follows a Gaussian distribution. Hence, does it imply that each margin will be a Gaussian distribution as well? "I understand that mixture components are used to describe the dependency structures, and they are the joint distribution. is that correct?

2- I read this from one answer to a question in this site here. "So yes, the assumption of joint normality is a sufficient condition for all marginal distributions to be normal, irrespective of the dependence structure. Hence, the theory of Copulas does not affect this result in any way."

However, I really do not understand this: irrespective of the dependence structure. As I understand if the joint distribution in normal then the dependence structure is normal.

Could someone help me with my questions, please?

Maryam
  • 1,012
  • 1
  • 5
  • 20

1 Answers1

8

Multivariate Gaussian mixtures are not themselves multivariate Gaussian, their components are.

Statements that apply to the components of a mixture don't generally apply to the mixture (this would be a fallacy of composition, a bit like saying "you have mixed red and blue, which are both primary colours so what you get must be a primary colour").

Multivariate Gaussian mixtures don't have normal margins except in a few special/degenerate cases (like a single component, or where you have an infinite number of components where the means are drawn from a normal distribution and the covariance matrix is constant across components).

As an example, consider a 50-50 mixture of two unit-variance uncorrelated-bivariate-normal components with mean at (-2,-2) and (2,2) respectively. Both margins are bimodal (indeed they have the same distribution in this example); clearly they aren't Gaussian.

plot of large sample from (X,Y) and histogram of marginal

Glen_b
  • 257,508
  • 32
  • 553
  • 939
  • Thank you so much for your clear answer. Is there a reasons that the normal assumption of the joint implies the that all marginals are normal does not apply to the mixture models? Also, could you please help me with the second question? – Maryam Jul 31 '18 at 05:12
  • The first question in your comment is completely addressed by the first 8 words of my answer. The part of your post under your point "2." doesn't actually contain any questions (there's nothing to answer there), but I believe your lack of understanding there is again completely covered by the first 8 words of my answer. Why would you expect properties of multivariate normals to hold for things that are not multivariate normal? – Glen_b Jul 31 '18 at 11:25
  • Thank you so much for your nice comment. My supervisors told me that Gaussian models can model the dependences structures of non-linear and also can be deal with non-gaussian margins and the assumption of normality does not make sense. – Maryam Jul 31 '18 at 12:27
  • If you mean Gaussian mixture models (which are not themselves Gaussian models), then yes, they can model nonlinear dependence and they do have non-Gaussian margins (as you see in the simple example in the answer). – Glen_b Jul 31 '18 at 12:54
  • But this source, https://www.researchgate.net/publication/259507449_Pair-copula_based_mixture_models_and_their_application_in_clustering, said GMM only assumed gaussian margins. (please see the introduction). – Maryam Jul 31 '18 at 12:58
  • Indeed they say just what you claim: "For example, each component follows a Gaussian distribution in case of GMM. Then each margin follows a univariate Gaussian distribution." ... but that statement is clearly false for an ordinary Gaussian mixture. I'm hoping that their intent is something other than how it looks there. It could potentially make some sense if they used a mixture to get the dependence structure and then built a copula from that on which they then placed a Gaussian margin, but that's not what they're discussing there; they're simply describing Gaussian mixtures ... ctd – Glen_b Jul 31 '18 at 13:25
  • ctd.. as far as I can see, and as I have explained multivariate Gaussian mixtures simply don't have Gaussian margins in general, only in a few special cases (and I clearly demonstrated an example where they didn't). Whatever they meant to say, they seem to have led you into some confusion. – Glen_b Jul 31 '18 at 13:29