Calculate the variance from variances

Question

Let's say I have three groups of values where each group has the same number of values. However, it is unknown how many values there are per group (the values are not available anymore). For each group I do have available the mean and the variance. How can I calculate the mean and the variance from the total population? For the mean that should be easy: It is simply the mean of the means. But how about the variance?

Edit: How to calculate the variance of a partition of variables seem to deal with a similar issue.

Answer 1

First of all, the mean is not exactly the mean of the means. But, considering $N=n_1+n_2+n_3$ the population (that in this sense is the union of the the three groups) average is $\mu=\frac{n_1\mu_1+n_2\mu_2+n_3\mu_3}{n_1+n_2+n_3}$. Thus, you have a set of averages in a simplex generated by the constraints $N=n_1+n_2+n_3$ and $n_1,n_2,n_3>0$. For the variance ($\sigma^2$), you can use a similar approach. The population variance is the sum of the Between Group Variance and the Within Group Variance as follows: $$N\cdot \sigma^2=\sum\limits_{g=1}^3 n_g(\mu_g-\mu)^2+\sum\limits_{g=1}^3 n_g \sigma^2_g$$ Also in this case, considering that $$\sum\limits_{g=1}^3 n_g(\mu_g-\mu)^2=\sum\limits_{g=1}^3 n_g\mu_g^2-N\cdot\mu^2$$ your solution is one of the possible inside the simplex. But remember that $\mu$ and $\sigma$ depends both on the choice of $n_1$,$n_2$, and $n_3$. In your case, $n_1=n_2=n_3$ the total variance is $$\sigma^2=\frac{1}{3}\sum\limits_{g=1}^3 \left[(\mu_g-\mu)^2+\sigma^2_g\right]$$

Answer 2

Well, variance estimation can be obtained for two groups (for simplicity) as follows:

${\hat{\sigma}^2} = \frac{1}{2N}\sum_{i=1}^{2N}{(X_i-\mu)^2 = \frac{1}{2N}\sum_{i=1}^{N}{(X_i-\mu)^2} + \frac{1}{2N}\sum_{i=N+1}^{2N}{(X_i-\mu)^2}=\frac{1}{2}(\hat{\sigma}^2_1 + \hat{\sigma}^2_2}) = \hat{\sigma}^2$

where ${X_i}$ - is a random variable (values in your case, which are not available anymore) and ${\mu}$ - is a mean.

so, variance of the total population is average of variances for every group ${\frac{1}{2}(\hat{\sigma}^2_1 + \hat{\sigma}^2_2)}$ where ${\hat{\sigma}^2_{1}}$ is a variance of group 1, the same for group 2.

quick test on Octave, where ${x, y}$ - are two groups:

octave:1> x = 3*randn(1000, 1);
octave:2> y = 3*randn(1000, 1);
octave:3> var(x)
ans =  9.0051
octave:4> var(y)
ans =  8.8170
octave:5> 0.5*(var(x) + var(y))
ans =  8.9111
octave:6>

${\hat{\sigma}^2_{1} = 9.0051}$, ${\hat{\sigma}^2_{2} = 9.0051}$, ${\hat{\sigma}^2 = 8.9111}$

Think of your estimation as a random variable, it has it's own mean and variance.

[EDIT] there is a better answer (and more correct).

Answer 3

I'm continue to simply Mr antonio irpino's answer in this answer, and note $\sum\limits_{i=1}^cn_i=n$ in here:

$$\begin{align}n\cdot \sigma^2&=\sum\limits_{i=1}^c n_i(\mu_i-\mu)^2+\sum\limits_{i=1}^c n_i \sigma^2_i\\ &=\sum\limits_{i=1}^c{n_i[\sigma^2_i+(\mu_i-\mu)^2]}\\ &=\sum_{i=1}^c{n_i(\sigma_i^2+\mu_i^2-2\mu \mu_i +\mu^2)}\\ &=\sum_{i=1}^c{n_i(\sigma_i^2+\mu_i^2)}+\sum_{i=1}^c{n_i(\mu^2-2\mu \mu_i)}\\ &=\sum_{i=1}^c{n_i(\sigma_i^2+\mu_i^2)}+\mu^2\sum_{i=1}^c{n_i}-2\mu\sum_{i=1}^c{n_i \mu_i}\\ &=\sum_{i=1}^c{n_i(\sigma_i^2+\mu_i^2)}+n \mu^2 -2n \mu^2\\ &=\sum\limits_{i=1}^cn_i(\sigma^2_i+\mu^2_i)-n\mu^2 \end{align}$$

conclusion:

It shows that we have the same symmetric structure for mean and variance in this case:

$$\begin{align}n&=\sum_{i=1}^c n_i\\ n\mu &=\sum\limits_{i=1}^{c}{n_i \mu_i}\\ n( \sigma ^2+\mu ^2 ) &=\sum_{i=1}^c{n_i( \sigma _i^2+\mu _i^2 )} \end{align}$$

Such a beautiful conclusion surprised me that I was the one who found it. At least I hadn't seen it anywhere else. It has quite the extreme beauty of Maxwell's system of equations. :)