Implications of i.i.d. sample

Question

I have the following question:

I have managed to solve it but I wasn't sure if my reasoning was correct. So I can express the OLS estimator as

$\sqrt{n}(\hat{\beta} - \beta) = (\frac{1}{n}\sum_{=1}^{n} x_{i}^2)^{-1}(\frac{1}{\sqrt{n}}\sum_{=1}^{n} x_{i}u_{i})$

By the weak law of large number it is known that the term in the first brackets approaches $E(x_{i}^2)$ but the distribution of the second term is where my confusion lies. I claimed that the second term approaches (in distribution) to a normal with mean zero and variance $E(x_{i}^2 u_{i}^2)$ because $E(x_{i}u_{i}) = 0$.

My question is: Is $E(x_{i}u_{i}) = 0$ because we assumed an i.i.d. sample or is it zero because $E(u_{i}|x_{i}) = 0$? My initial guess was that since the sample is i.i.d., there should be no covariance between $x_{i}$ and $u_{i}$ and thus $E(x_{i}u_{i}) = 0$. But is this reasoning correct?

On an unrelated note, if anyone knows of any books that has similar questions to this one (i.e deriving asymptotic properties of different estimators), please let me know.

Answer 1

$E(x_{i}u_{i}) = 0$ is indeed implied by an assumption like $E(u_{i}|x_{i}) = 0$, thanks to the law of iterated expectations, $$ E(x_{i}u_{i})=E[E(x_{i}u_{i}|x_i)]=E[x_{i}E(u_{i}|x_i)]=E[x_{i}0]=E[0]=0. $$

Assuming random sampling does not imply $E(x_{i}u_{i}) = 0$. Random sampling basically tells us that "neighboring" values $(x_j,y_j)$, $i\neq j$, contain no additional information on $(x_i,y_i)$. E.g., if you randomly sample from the population of American households, knowing that household 41,234,678 is, say, unusually rich will not make you believe that 41,234,679 will be so, too. Indeed, under random sampling, there will be no special connection whatsoever between neighboring households in the sample.

Things may look quite different in a time series setting. Here, the neighbors in the dataset are the observations from the recent past/future, and it is often quite plausible (indeed, the raison d'être of time series analysis) that, e.g., there are several consecutive quarters of above-average economic growth ("business cycles").

What $E(u_{i}|x_{i}) = 0$ tells you is that there is (recall that this is an assumption that is neither easy to test, as we do not observe $u_i$, nor easy to satisfy, as "correctly specifying" a model is difficult!) no relationship between regressor and errors, so, that we did not forget important variables in the specification of our regression model.

If we do forget such important variables, we have omitted variable bias, see e.g. here for examples: Observational vs quasi-experimental design?

You may also have encountered the statement that we need $E(u|X)=0$, where the arguments now denote vectors/matrices, to obtain that OLS is unbiased. Here, we indeed obtain a change to that statement for i.i.d. data, in that it is not necessary to condition on "neighbors" anymore, so that the expression simplifies to $E(u_{i}|x_{i}) = 0$.

You will find a related discussion here: Proving OLS unbiasedness without conditional zero error expectation?

In my opintion, a useful reference is Hayashi, Econometrics.