Questions tagged [mixture-distribution]

A mixture distribution is one that is written as a convex combination of other distributions. Use the "compound-distributions" tag for "concatenations" of distributions (where a parameter of a distribution is itself a random variable).

Mixture models arise in attempts to characterize complicated probability distributions, especially those with two or more modes, in terms of distributions with mathematically simple descriptions.

Disambiguation

  • Do not confuse a "mixture model" with a "mixed model"! The former concerns distributions, typically multi-modal, that will be analyzed as positive linear combinations of other distributions. The latter occurs in a regression setting where some of the independent variables are viewed as fixed and others are viewed as realizations of random variables.

  • Note that although the density of a mixture is, by definition, a linear combination of densities, it is not in general the same as the density of a linear combination of random variables. For example, the average of two normal random variables is normal (and therefore has a single mode), but a 50:50 mixture of two different normal densities often has two modes and is never normal.

  • Compound distributions are also known as "mixtures". Please use the tag in such cases. See the meta thread on The “mixture” vs. the “compound-distributions” tags for details.

449 questions
60
votes
2 answers

What is the variance of the weighted mixture of two gaussians?

Say I have two normal distributions A and B with means $\mu_A$ and $\mu_B$ and variances $\sigma_A$ and $\sigma_B$. I want to take a weighted mixture of these two distributions using weights $p$ and $q$ where $0\le p \le 1$ and $q = 1-p$. I know…
JoFrhwld
  • 2,247
  • 3
  • 20
  • 22
54
votes
10 answers

Why is the sum of two random variables a convolution?

For long time I did not understand why the "sum" of two random variables is their convolution, whereas a mixture density function sum of $f(x)$ and $g(x)$ is $p\,f(x)+(1-p)g(x)$; the arithmetic sum and not their convolution. The exact phrase "the…
32
votes
3 answers

Student t as mixture of gaussian

Using the student t-distribution with $k > 0$ degrees of freedom, location parameter $l$ and scale parameter $s$ having density $$\frac{\Gamma \left(\frac{k+1}{2}\right)}{\Gamma\left(\frac{k}{2}\sqrt{k \pi s^2}\right)} \left\{ 1 + k^{-1}\left(…
28
votes
4 answers

Generating random variables from a mixture of Normal distributions

How can I sample from a mixture distribution, and in particular a mixture of Normal distributions in R? For example, if I wanted to sample from: $$ 0.3\!\times\mathcal{N}(0,1)\; + \;0.5\!\times\mathcal{N}(10,1)\; +…
user30490
21
votes
5 answers

Motivation of Expectation Maximization algorithm

In the EM algorithm approach we use Jensen's inequality to arrive at $$\log p(x|\theta) \geq \int \log p(z,x|\theta) p(z|x,\theta^{(k)}) dz - \int \log p(z|x,\theta) p(z|x,\theta^{(k)})dz$$ and define $\theta^{(k+1)}$ by $$\theta^{(k+1)}=\arg…
user782220
  • 952
  • 9
  • 15
21
votes
4 answers

Maximum likelihood function for mixed type distribution

In general we maximize a function $$ L(\theta; x_1, \ldots, x_n) = \prod_{i=1}^n f(x_i \mid \theta) $$ where $f$ is probability density function if the underlying distribution is continuous, and a probability mass function (with summation instead…
17
votes
2 answers

" all of these data points come from the same distribution." How to test?

I feel like I've seen this topic discussed here before, but I wasn't able to find anything specific. Then again, I'm also not really sure what to search for. I have a one dimensional set of ordered data. I hypothesize that all of the points in the…
shadowtalker
  • 11,395
  • 3
  • 49
  • 109
17
votes
2 answers

Is there a standard method to deal with label switching problem in MCMC estimation of mixture models?

Label switching (i.e., the posterior distribution is invariant to switching component labels) is a problematic issue when using MCMC to estimate mixture models. Is there a standard (as in widely accepted) methodology to deal with the issue? If…
user28
15
votes
6 answers

Time spent in an activity as an independent variable

I want to include time spent doing something (weeks breastfeeding, for example) as an independent variable in a linear model. However, some observations do not engage in the behavior at all. Coding them as 0 isn't really right, because 0 is…
D L Dahly
  • 3,663
  • 1
  • 24
  • 51
14
votes
2 answers

Proper use and interpretation of zero-inflated gamma models

Background: I am a biostatistician presently wrestling with a dataset of cellular expression rates. The study exposed a host of cells, collected in groups from various donors, to certain peptides. Cells either express certain biomarkers in response,…
14
votes
3 answers

Relation between sum of Gaussian RVs and Gaussian Mixture

I know that a sum of Gaussians is Gaussian. So, how is a mixture of Gaussians different? I mean, a mixture of Gaussians is just a sum of Gaussians (where each Gaussian is multiplied by the respective mixing coefficient) right?
13
votes
2 answers

Why is clutter problem intractable for large sample sizes?

Suppose we have a set of points $\mathbf{y} = \{y_1, y_2, \ldots, y_N \}$. Each point $y_i$ is generated using distribution $$ p(y_i| x) = \frac12 \mathcal{N}(x, 1) + \frac12 \mathcal{N}(0, 10). $$ To obtain posterior for $x$ we write $$ p(x|…
Alexey Zaytsev
  • 2,335
  • 15
  • 26
13
votes
1 answer

Python packages for working with Gaussian mixture models (GMMs)

There seem to be several options available for working with Gaussian Mixture Models (GMMs) in Python. At first glance there are at least: PyMix: Tools for mixture modeling PyEM [dead link], which is part of the Scipy toolbox and seems to focus on…
Aman
  • 1,463
  • 13
  • 13
13
votes
1 answer

Marginal Likelihood from the Gibbs Output

I'm reproducing from scratch the results in Section 4.2.1 of Marginal Likelihood from the Gibbs Output Siddhartha Chib Journal of the American Statistical Association, Vol. 90, No. 432. (Dec., 1995), pp. 1313-1321. It's a mixture of normals model…
Zen
  • 21,786
  • 3
  • 72
  • 114
12
votes
2 answers

What is the distance between a finite Gaussian mixture and a Gaussian?

Suppose I have a mixture of finitely many Gaussians with known weights, means, and standard deviations. The means are not equal. The mean and standard deviation of the mixture can be calculated, of course, since the moments are weighted averages of…
Douglas Zare
  • 10,278
  • 2
  • 38
  • 46
1
2 3
29 30