4

I have read the other posts about standard error and the proof is neat and I am able to follow, however, since I am new to statistics, there's some gap in logic and concept in between the proof.

Here is the link that I closely follow on general method for deriving the standard error

Below is my attempt to reconstruct the proof, but I get very confused over the definition of random variable. In particular, given a population which consist of all the height of the people in California which follows a certain distribution with variance $\sigma^2$, then we can define a random variable $X$ such that $X$ takes on all the heights of the people of California. Denote each observation in $X$ to be $X_i, 1 \leq i \leq n$, is it true that each $X_i$ is also a random variable following the same variance? You will see why I am confused with the proof given below by me.

Since we are proving the derivation of standard error of the mean. Let the sample mean be a random variable called $\bar{X}$, we aim to find the variance of this $\bar{X}$. For any $\bar{X}$, it is merely equals to $\frac{1}{n}\sum_{i=1}^{n}X_i$ where $n$ is the size of the sample and $X_i, 1 \leq i \leq n$ is a single observation from the population. Here we can say that each $X_i$ is a random variable as well which is independent and following a population variance of $\sigma^2$. Since $$\bar{X} = \frac{1}{n}\sum_{i=1}^nX_i$$ it hence follows that

$ \begin{aligned} \text{Var}(\bar{X}) &= \text{Var}\left(\frac{1}{n}\sum_{i=1}^nX_i\right) \\ & =\left(\frac{1}{n}\right)^2\sum_{i=1}^{n}\text{Var}(X_i)\\ & = \left(\frac{1}{n}\right)^2(n\sigma^2)\\ &= \frac{\sigma^2}{n}\\ \end{aligned} $

nan
  • 511
  • 2
  • 9
  • Hi: what you did is correct, if that's what you're asking. – mlofton Sep 07 '19 at 12:44
  • @mlofton Hi thanks for the reply, what I'm confused about is since each $X_i$ is just a value from the population, why is it modeled as a random variable? – nan Sep 07 '19 at 13:30
  • Hi: Think of it as if someone picked these particular $X_i$, $i = 1, \ldots n $ randomly from some population which is gigantic as in billions. Each individual element contained in the billion is every single X_{i}. But you just see a sample of size $n$. Each individual observation that you see is an RV because it originated from the population. The population is represented mathematically by some distribution. The distribution is just a fancy way of saying what a histogram would look like if you constructed a histogram using the every single element in the population. – mlofton Sep 07 '19 at 19:27
  • 1
    "Here we can say that each $X_i$ is a random variable" Yes we can, and all of the $X_i$ have the same mean $\mu$ and the same variance $\sigma^2$ where $\mu$ and $\sigma^2$ are the population mean and population variance. Where I strongly disagree with your sentence is the part which claims that the $X_i$ are _independent_: they are independent only if the sampling is with replacement and not independent if the sampling is without replacement. The latter scenario is what you seem to envisioning in your sampling of the heights of the population of California. – Dilip Sarwate Sep 07 '19 at 22:45

1 Answers1

1

Conceptually, population represents the distribution of values that the random variable can take. The random variable, $X$, represents the value of a randomly chosen sample from this population. In this problem, it is the height of a random person among Californians. We could as well choose two people, call them as $X,Y$ and analyze their properties, e.g. mean, max etc. We could have named those RVs as $X_1,X_2$, while nothing conceptually changes. Or, we could pick $n$ people from the population and name them $X_1,X_2,...,X_n$. These are still RVs. You can have data for these $n$ people, but still it doesn't prevent them from being modeled as RVs. The data might be considered as a realization of this RV collection. It's like tossing a coin $n$ times and having $X_1,X_2,...,X_n$ as your outcomes.

gunes
  • 49,700
  • 3
  • 39
  • 75
  • Is there a difference between an “observation” from the population and the “random variable”? An individual observation from the population which is denoted as $X_1$ can be modeled as a RV as wel? – nan Sep 07 '19 at 18:11
  • Yes, you can model. For example, you have an observation $X_1=a$, e.g. somebody's height. When we treat $X_1$ as a RV drawn from the population, i.e. distributed wrt $f_X(x)$, we can calculate the odds of having this specific value, e.g. what is the probability of a chosen person (person $i$) having height $a$ meters, for example. – gunes Sep 07 '19 at 18:18
  • Thank you for the clear explanation. One last confusion is the last paragraph of my question. For $$\bar{X} = \frac{1}{n}\sum_{i=1}^nX_i$$ where $\bar{X}$ is the sample mean, in this scenario, is our $X_i$ a random variable or an observation value from the sample...? Or since you have explained, it can be both. – nan Sep 07 '19 at 18:22
  • You're welcome. Yes, it can be both. Because, once you have data you can simply evaluate the sample mean. Having a sample mean value doesn't mean that we can't treat its definition as a RV. Sample mean has also a distribution. Because, another $n$ sample drawn from the population would create another sample mean. So, it can change from experiment to experiment. – gunes Sep 07 '19 at 18:25