1

I have been doing some reading on standard errors and the derivation of that formula. Unfortunately I have hit a snag in my understanding of a variance. Specifically, I am having a hard time understanding the difference between these two notations:

First from the question titled "General method for deriving the standard error", I see the following formula which seems to jive with the derivation found on Wikipedia (Standard Error / Derivations)

$$ Var(\sum\limits_{i=1}^nX_i) =\sum\limits_{i=1}^nVar(X_i)=\sum\limits_{i=1}^nσ^2=nσ^2 $$

Second, the other formula I'm seeing is the standard variance formula:

$$ Var(X) = \frac{1}{n} \sum\limits_{i=1}^n(x_i - \mu)^2 $$

Can someone help me understand the difference in these two equations? The point where I am having an issue is the random variable being operated upon. In the first equation we see an indexed variable. Why do we not see that in the second equation? Ultimately in the context of standard errors of the mean, How do these two definitions relate?

samvoit4
  • 185
  • 1
  • 8
  • You have changed the meaning of "$X$" in the two formulas. In the first, it's a vector-valued random variable $(X_1,X_2,\ldots,X_n)$; in the second, it's a vector of numbers, $(x_1,x_2,\ldots,x_n).$ – whuber Jun 06 '18 at 21:05
  • It looks to me that in the second case X is the sample mean while in the first case the ransom variable is a sum. Are we dealing with i. i. d. random variables with a finite variances? – Michael R. Chernick Jun 06 '18 at 21:24
  • The Wikipedia reference is referring to i.i.d random variables with mean $\mu$ and variance $\sigma^2$. – Michael R. Chernick Jun 06 '18 at 21:28
  • @sam can you give a reference for that second formula? It looks odd; it has the form of a sample variance but looks at deviations from the population mean. While such a formula can crop up occasionally (I've used it myself in explaining why the sample variance with $n$ denominator is biased downward), it's not a universal formula and I wouldn't expect to see it for something you'd normally want to use. – Glen_b Jun 06 '18 at 22:15
  • @Glen_b I am using the population variance found here: [Statistics How To](http://www.statisticshowto.com/probability-and-statistics/variance/) – samvoit4 Jun 07 '18 at 00:16
  • Okay; sure, that's a population variance for a finite population with n elements in the population. It's not much use for infinite populations (and many populations are notionally infinite or treated as infinite -- including every time someone claims a population is normally distributed). – Glen_b Jun 07 '18 at 01:53

1 Answers1

1

This formula: $Var(X) = \frac{1}{n} \sum\limits_{i=1}^n(x_i - \mu)^2$ is the formula for the population variance in a finite population. It can be seen as the variance of a variable with a discrete distribution over $\{x_1, x_2, ..., x_n\}$ each with probability $p_i=\frac{1}{n}$.

In other circumstances you'll have other formulas for population variance

(e.g. see the formulas under https://en.wikipedia.org/wiki/Variance#Definition ... though these are all basically versions of the same underlying formula)

Now imagine I have two populations (whether finite or infinite, and if infinite, whether discrete, continuous or mixed); and I draw pairs of elements independently from each and add their values (e.g. say the first is the weight of a particular kind of manufactured bottle and the second is the weight of the liquid in it, and I want to know about the combined weight).

Then how do I find the variance of the variable that results from adding two independent variables?

I use the fact that the variance of the sum is the sum of their variances.

This is your first formula. The "n" in that formula isn't a population size; it's how many different variables are in your sum (in the case of my bottle + liquid example that $n=2$).

Now when all the variables have the same variance (perhaps because they're all drawn with replacement from the same population) then the variance of the sum is just the sum of $n$ lots of the common variances, $\sigma^2$, so "the sum of the variances" would then be $n\sigma^2$.

Glen_b
  • 257,508
  • 32
  • 553
  • 939
  • Thank you for your response! This makes sense to me. One clarifying question: when referring to the standard error of the mean, we are sampling a population many times and recording the means of our samples. In this case is the "n" the number of samples we have taken (i.e. the number of means we have recorded)? Does the random variable X_i represent the samples we have examined from our population? I think that's what you are saying in your last paragraph, but I want to confirm. – samvoit4 Jun 07 '18 at 13:28