0

I need to calculate the average and standard deviation of a population. However, I only have the information of sample means and standard deviations.

To be more precise, I have a vector of means $\bar{\mathbf{x}}=(\bar{x}_1, \bar{x}_2, \ldots, \bar{x}_N)$ with corresponding vector $\mathbf{n}=(n_1, n_2, \ldots, n_N)$ of sample sizes and a vector of standard deviations $\mathbf{s}=(s_1, s_2, \ldots, s_N)$. I need to compute the overall average and standard deviation.

To obtain the overall mean, I calculated it using the formula $$\bar{x}=\dfrac{\sum_{i=1}^N\bar{x}_in_i}{\sum_{i=1}^Nn_i}.$$ Whereas, for the overall standard deviation, I can compute it from the overall deviation $$Var=\frac{1}{n−1}\left(\sum_{j=1}^N(n_j−1)V_j+\sum_{j=1}^Nn_j(\bar{x}_j−\bar{x})^2\right)\,\;,$$ where $V_j$ is the deviation of the of the $j-$th sample.

Is there a way that I could compute these values in a "online" / "streaming way" (e.g., when the vectors are infinite) using only the aforementioned vectors?

Thanks in advance

Note that this question can be seem as complement to:

  • https://stats.stackexchange.com/questions/216047/how-does-one-go-about-determining-the-standard-deviation-of-an-entire-sample-dat, where the above formula for deviation is presented and
  • https://stats.stackexchange.com/questions/72212/updating-variance-of-a-dataset, where the mean and deviation are update at each new observation. In my case, I do not have access to the observations.
ℂybernetician
  • 113
  • 7
  • $1$. why do you have bars on your n's and s's? $2$. This question is already answered on site, for example, here: https://stats.stackexchange.com/questions/216047/how-does-one-go-about-determining-the-standard-deviation-of-an-entire-sample-dat/216060#216060 with pointers to other answers to the two subproblems (i.e. (i) calculating overall variance from subgroup means, variances or standard deviations and sample sizes and (ii) for online updating of variance) with some discussion of how they combine. – Glen_b Aug 08 '19 at 03:47
  • If you need additional clarification, ask something specific relating to what you can't get from the existing answer (you can post a new question if it's not a simple clarification of the present one) – Glen_b Aug 08 '19 at 03:57
  • Hi @Glen_b, thanks for the heads up. I have updated the question formulation. – ℂybernetician Aug 08 '19 at 04:28

0 Answers0