0

What's the derivation for expected value for sample variance for a sample taken from simple random sampling without replacement, i.e., how do we show that

$$\mathrm{E}(s^2) = \sigma^2 \frac{N}{N-1}$$

Is my assumption this only applies to SRS samples without replacements correct?

On a related note, how is the sample variance from two numbers, $x_1, x_2$ equal to $((x_1-x_2)^2)/2$

The same notation is used in this webpage/doc, http://www.utdallas.edu/~serfling/handouts/ExpValSampVar.pdf

COOLSerdash
  • 25,317
  • 8
  • 73
  • 123
DJ_
  • 743
  • 3
  • 12
  • 23
  • This appears to fall under the [self-study](http://stats.stackexchange.com/tags/self-study/info) tag. please add it here and on your other question(s) as needed. – Glen_b Oct 02 '13 at 03:27

2 Answers2

3

Suppose your population is $\{x_1,...,x_N\}$. The sample without a replacement of a size $m$ is $\{x_{i_1},...,x_{i_m}\}$, where indexes $\{i_1,...,i_m\}$ form a subset of the index set $\{1,...,N\}$. Now there are $m \choose n$ ways of selecting such a subset so your sample has a probability $1/{m \choose n}$ of being drawn. Now

$$Es^2=\sum_{i_1,...,i_m\subset {1,...,N}}s_{i_1,...,i_m}^2\frac{1}{m\choose n},\quad(1)$$

where

$$s^2_{i_1,...,i_m}=\frac{1}{m}\sum_{k=1}^mx_{i_k}^2-\left(\frac{1}{m}\sum_{k=1}^mx_{i_k}\right)^2, \quad (2)$$

Now you must substitute (2) into (1) and show that the expression is equal to

$$\frac{N}{N-1}\left(\frac{1}{N}\sum_{k=1}^Nx_{k}^2-\left(\frac{1}{N}\sum_{k=1}^Nx_{k}\right)^2\right),$$

where I substituted the definition of $\sigma^2$. The task might look daunting, but it is not that complicated. Here are two hints:

$$\sum_{i_1,...,i_k}x_{i_k}^2=\sum_{k=1}^Nx_k^2 {{m-1}\choose {N-1}}$$

$$\sum_{i_1,...,i_k}x_{i_k}x_{i_l}=\sum_{k\neq l}x_kx_l{{m-2}\choose {N-2}}$$

There is also less complicated way of achieving that, but I forgot the details how, I took the survey statistics curse 10 years ago and did not use them afterwards.

mpiktas
  • 33,140
  • 5
  • 82
  • 138
1

I originally answered this question incorrectly, but I will leave my answer.

The sample variance (with replacement) is defined to be a unbiased estimator of the variance. Therefore $E(s^2)=\sigma^2$. There are many examples online. Here is a good reference:

http://pascencio.cos.ucf.edu/classes/Methods/Proof%20that%20Sample%20Variance%20is%20Unbiased.pdf

On your related note, I don't follow your notation. Please see the following reference for a clear proof on the variance of the difference of two random variables.

http://www.sp.uconn.edu/~st100is1/pdf/Ch11les.pdf

phubaba
  • 213
  • 1
  • 6
  • This is not correct, also this is a webpage that uses the same notation however I don't get their derivation, http://www.utdallas.edu/~serfling/handouts/ExpValSampVar.pdf – DJ_ Oct 02 '13 at 01:19
  • 2
    @DJ_ Both are correct. Your expression is the expected value of the sample variance. However, this value is biased. That's why in order to obtain an unbiased estimator, you need to multiply $\sigma^{2}$ by $N/(N-1)$. This is the value that the proof given by phubaba is using and its purpose is precisely to allow $E[s^{2}] = \sigma^{2}$ – Robert Smith Oct 02 '13 at 04:31
  • 2
    (-1) This answer uses "consistent" where "unbiased" is needed: see http://stats.stackexchange.com/questions/31036. As such it only repeats the statement that is to be proved and sows additional confusion. – whuber Oct 02 '13 at 14:49
  • thanks whuber, sometimes answering a question is the best way to realize that you don't actually know the answer either. appreciate the comment – phubaba Oct 03 '13 at 13:04
  • 2
    @phubaba you can delete your own answer. – mpiktas Oct 03 '13 at 13:18
  • 1
    phubaba, I appreciate the sentiment--I participate regularly here because I learn so much from my own mistakes and misconceptions. The purpose of these comments (and the downvote) is to encourage you to use what you have learned to correct and improve your answer. This would be preferable to deleting it, because you have researched some apparently useful links. – whuber Oct 03 '13 at 13:33