4

What is wrong with the following sentence:

The Central Limit Theorem implies that, as the sample size grows, the error distribution approaches normality.

Am I correct by saying that it should in stead state the MEAN of the sample error approaches zero as sample size grows?

Las
  • 53
  • 1
  • 3
  • I meant to say "the distribution of the mean of the sample error approaches zero" Do you agree? Or am I missing the mistake in the statement completely? Thanks for the opinions (in advance) – Las Feb 09 '12 at 16:07
  • 3
    You're going to need to be more specific (e.g. the error distribution of...? What model are you using?) but the second thing you said sound closer to the law of large numbers than the CLT. – Macro Feb 09 '12 at 16:23
  • The error distribution does not change. – Xi'an Feb 09 '12 at 19:00
  • 2
    It depends on what "error distribution" means, Xi'an (and what statistic it refers to, and what kind of asymptotic setting one is in, etc.). This question is impossible to answer without making (at least some implicit) assumptions about the context. It would help for the OP to respond to Macro's request for more information. – whuber Feb 09 '12 at 20:02

1 Answers1

14

In its standard simplest form, the Central Limit Theorem (CLT) is a statement about the cumulative distribution function of the random variable $$Z_n = \frac{X_1 + X_2 + \cdots + X_n -n\mu}{\sigma \sqrt{n}}$$ where the $X_i$ are independent identically distributed random variables with mean $\mu$ and standard deviation $\sigma$. The CLT asserts that for each $a$, $-\infty < a < \infty$, $$F_{Z_n}(a) = P\left\{\frac{X_1 + X_2 + \cdots + X_n -n\mu}{\sigma \sqrt{n}} \leq a \right\} \to \Phi(a) = \int_{-\infty}^a \frac{e^{-x^2/2}}{\sqrt{2\pi}}\mathrm dx$$ as $n \to \infty$.

If by "error distribution" you mean the distribution function of $$Y_n = \left(\frac{1}{n}\sum_{i=1}^n X_i\right) -\mu = \frac{\sigma}{\sqrt{n}}Z_n,$$ that is, the difference of the sample mean $\bar{X} = n^{-1}\sum_iX_i$ and the population mean $\mu$, then the CLT certainly does not imply that $F_{Y_n}(\cdot)$ "approaches normality" as the sample size $n$ grows large in the usual sense of normality, though nitpickers may want to claim that the distribution is approaching a normal distribution with mean $0$ and standard deviation $0$ (often called a constant by statistically illiterate people).

On the other hand, the mean of the sample error is not a random variable but a constant (in fact, $0$ since the sample mean is an unbiased estimator of the population mean) and does not need to approach $0$; it is already there! I think what you meant to say is that the distribution $F_{Y_n}(a)$ of the sample error approaches the unit step function: $$F_{Y_n}(a) \to u(a) = \begin{cases}1, & \text{if}~a > 0,\\ 0 &\text{if}~a < 0,\end{cases}$$ which is certainly correct, and follows from the CLT, but also follows from results such as the weak law of large numbers which makes no assertions about the distribution of $Z_n$, only about $Y_n$.

Dilip Sarwate
  • 41,202
  • 4
  • 94
  • 200
  • 2
    (+1) Here is a **fun** nitpick: One needs the extra assumption that $u(a)$ is continuous at $a$ to be able to conclude $F_{Y_n}(a) \to u(a)$, so, in particular, the first inequality ($a \geq 0$) also needs to be strict ($a > 0$), like the second one. A canonical example is to take $X_i$ iid uniform on $\{-1,+1\}$ and show that $F_{Y_n}(0) \to 1/2$. Showing this for the subsequence of *odd* $n$ is easiest (just symmetry), but it's not much harder to show it holds also for the subsequence of even $n$. – cardinal Feb 09 '12 at 19:36
  • @cardinal You are right; in the interest of brevity, I had skipped stating the part where convergence is guaranteed only at points where $u(\cdot)$ is continuous, and then written the standard CDF description. Thanks for pointing out the mistake. I will fix the statement. – Dilip Sarwate Feb 09 '12 at 19:48
  • This is all thew information I was given (old test question). But we focus on OLS models if that makes a difference at all. Thanks for the information. – Las Feb 10 '12 at 15:39
  • @Las You and user9097 (cf. [this question](http://stats.stackexchange.com/q/22557/6633)) seem to be looking at the same exam! If the question on the exam asks whether the statement "The CLT says the distribution of the sample mean asymptotically converges to a normal distribution" is correct, the answer is NO. If the exam goes on to ask "If you answered NO, what _does_ the CLT say actually?", the answer is that subject to suitable conditions (which you can get from the above), the CLT says that the distribution of $\sqrt{n}\bar{X}$ converge to a normal $\mathcal N(\mu, \sigma)$ distribution. – Dilip Sarwate Feb 10 '12 at 16:11
  • Thanks Dilip. That certainly makes more sense to me now. – Las Feb 10 '12 at 17:32
  • @DilipSarwate - Hello Sir! Can you please explain how the distribution of $\sqrt{n}\bar{X}$ converges to a normal $\mathcal{N}(\mu,\sigma)$. If by $\bar{X}$ you meant sample mean then would not the distribution be $\mathcal{N}(\sqrt{n}\mu,\sigma^2)$? – MiloMinderbinder Apr 11 '21 at 15:28