3

for a simple linear model:

$y= \alpha+\beta x+e$

$\hat \beta= \frac{cov(x,y)}{var(x)} = $ second element of $(X'X)^{-1}X'y $

where $X = [1, x]$

However, if the true relation is

$y= \beta x+e$ with $cov(x,e)=0$

the cov formula still works as

$\frac{cov(x,y)}{var(x)} = \frac{cov(x,(\beta x+e))}{var(x)} = \frac{\beta*cov(x,x)+cov(x,e)}{var(x)} = \beta $

But I am trying to understand why the least squares doesnt work anymore:

$(X'X)^{-1}X'y = (X'X)^{-1}X'(X\beta+e) = \beta + (X'X)^{-1}X'e = \ \beta$

But now for true model $(X'X)^{-1}X'y = \frac{\sum(xy)}{\sum(x^2)} \not= \frac{cov(x,y)}{var(x)} $ for model with a constant.

Can someone point out if I might be mistaken in thinking that regular OLS with constant is still unbiased and consistent? How can both of these be unbiased as they are different.

In other words, are both $\frac{\sum(xy)}{\sum(x^2)} $ and $ \frac{cov(x,y)}{var(x)} $ unbiased/consistent when true $\alpha=0$ and how (assuming $E(x), E(y) \not=0$)?

dayum
  • 469
  • 2
  • 10

1 Answers1

4

Yes, both are unbiased when $\alpha=0$. If the true model does not contain a constant, the regression that does fit one contains a superfluous regressor. This is not good for efficiency, but has no impact on unbiasedness.

For the opposite case in which the true model does contain a nonzero intercept but a regression is fit without one, see for example this post.

You do not distinguish very clearly between sample and population (co)variances, but note that, e.g., the population covariance is generally defined as $$ E[(X-E(X))(Y-E(Y))] $$ When $E(X)=E(Y)=0$, this evidently simplifies to $E[XY]$. So, in that sense, your final display is true in the case of a true model without constant.

Christoph Hanck
  • 25,948
  • 3
  • 57
  • 106
  • @Christopher_Hanck , yes exactly what I'm trying to understand. How are they both unbiased in case of nonzero means. E(X) or E(y) need not be zero in case of non constant, right? – dayum May 16 '18 at 22:38
  • 1
    See e.g. https://stats.stackexchange.com/questions/120731/does-adjusting-for-superfluous-variables-bias-ols-estimates – Christoph Hanck May 17 '18 at 08:04