Proof that predictions are unbiased in in endogenous linear model

Question

Problem Statement

Suppose we have a linear model given by $$y = X\beta + \varepsilon,$$ where $\varepsilon\sim N(0, \sigma^2 I)$ and $E[\varepsilon|X]\neq0$ (i.e., explanatory variables are endogenous). Let the OLS estimate of $\beta$ be denoted $\hat\beta = (X'X)^{-1}X'y$.

One can show that the endogeneity assumption implies $E[\hat\beta]\neq\beta$. However, I've seen the claim that predictions from models suffering from endogeneity remain unbiased. I'm trying to determine if this is true through proof/disproof of the following two claims.

Claim 1: $E[X\hat\beta|X] = X\beta (=E[y|X])$

I'm reasonably sure this is true since OLS chooses $\hat\beta$ to minimize sum of squared residuals. For the proof, I have begun with \begin{align} E[X\hat\beta|X] &= E[X(X'X)^{-1}X'y|X]\\ &=E[X(X'X)^{-1}X'(X\beta + \varepsilon)|X]\\ &=E[X(X'X)^{-1}X'X\beta|X] + E[X(X'X)^{-1}X'\varepsilon|X]\\ &=X\beta + E[X(X'X)^{-1}X'\varepsilon|X]. \end{align} However, I can't determine how the second term is zero since $E[\varepsilon|X]\neq0$.

Claim 2: $E[\hat{X}\hat\beta|X] = \hat{X}\beta$ for $\hat{X}\neq X$

I know this claim isn't true in general. For example, if $\hat{X} = (0, 1, 0, ..., 0)$, then $$E[\hat{X}\hat\beta|X] = E[\hat\beta_1|X] \neq \beta_1$$ in general since $\hat{\beta}$ is not an unbiased estimator of $\beta$.

However, I am interested in sufficient conditions under which this claim is true. For example, is it possible to show this claim is true if $E[X'\varepsilon] = E[\hat{X}'\varepsilon]$ or under some other assumption?

Answer 1

The answers are actually pretty straightforward. In general the predictions will not be unbiased, to see that just notice:

$$ E[y|X]= X\beta + E[\epsilon|X] $$

Thus, if $E[\epsilon|X]$ is not linear function of $X$, the population linear regression will not recover the true expectation function $E[y|X]$. Instead, it will give you the best linear approximation of $y$ (best as in minimizing the quadratic error). Since this is true for the population, of course sample estimates are also not consistent.

Analogously, if $E[\epsilon|X]$ is a linear function of $X$, then the population linear regression is by definition $E[y|X]$. So all standard results for linear regression apply, and you will get unbiased predictions. You just won't recover the structural $\beta$, instead you will recover the population regression coefficients $E[XX']^{-1}E[XY]$.

Answer 2

Claims 1 and 2 are actually both false. To show that they are false over a broad class of models in which $E[\epsilon|X] \neq 0$, it is enough to pick a single model or a narrow set of models from within that broad class of models, and to disprove the claims for the narrow set. I will follow this strategy in my answer.

Suppose the true model is $y_i = z_i\Gamma + x_i\beta + \delta_i$ for some zero-mean $\delta_i$'s and for a hidden set of confounders $Z$. Suppose $E[z_i|x_i] = x_iA$. Your $\epsilon_i$ is my $z_i\Gamma + \delta_i = x_iA\Gamma + \delta_i$. Then $E[y_i] = x_iA\Gamma + x_i\beta + \delta_i$, and the OLS estimates are targeting $\tilde\beta \equiv A\Gamma + \beta$ instead of just $\beta$. But, the true function is still linear, so predictions will be unbiased.

To address your claims directly, using my notation:

• Claim 1 should read $E[ X \hat \beta] = X\tilde\beta$, not $E[X \hat \beta ] = X\beta$. So as stated, it is false. Your second term, in my notation, is $A\Gamma$, which is indeed nonzero.
• Claim 2 is likewise missing a tilde.

Claim 1 is worth a little more discussion. If it were modified to include $A\Gamma$, it would become true, at least for the limited set of models I have discussed. This is significant because it is a formal statement of the earlier assertion that predictions are unbiased. To expand on this, it's worth stating explicitly what I mean by an unbiased prediction. The predictor that best avoids systematic differences with the true model is $X \beta+ E[\epsilon|X] = X \beta+ E[Z\Gamma + \delta|X] = X \beta+ X A\Gamma$. I would call unbiased any prediction method whose expectation is this. Since $\beta+ A\Gamma$ is exactly what the OLS estimates target, OLS produces unbiased predictions here. (By contrast, this is still biased if you want to infer $\beta$.)