2

Let $\gamma$ and $\delta$ denote $K\times 1$ vectors of parameters in models $\hat{Y} = X\gamma+\eta$ and $\hat{\epsilon} = X\delta+\psi$, where $\eta$ and $\psi$ and $n\times 1$ vectors of error terms. Show that the OLS estimators of $\gamma$ and $\delta$ are, \begin{align} \hat{\gamma} &= \hat{\beta}; \\ \hat{\delta} &= 0_{K\times1}; \end{align}

The definitions of $\hat{Y}$ and $\hat{\epsilon}$ are \begin{align} \hat{Y} &=X\hat{\beta} \\ \hat{\epsilon} &= Y-\hat{Y}. \end{align}

I understand that we can show the first one by the following,

$$\hat{\gamma} =(X'X)^{-1}X'\hat{Y} = \hat{\beta}$$

But I'm unable to show the second expression, any guidance or help would be highly appreciated.

Christoph Hanck
  • 25,948
  • 3
  • 57
  • 106
MisterButter
  • 187
  • 7

1 Answers1

4

The residuals $\hat{\epsilon}= Y-\hat{Y}$ can be written as $\hat{\epsilon}=MY$, where $M=I-X(X'X)^{-1}X'$ is the so-called "residual-maker matrix". Hence, using the OLS formula, we have $$ \hat\delta=(X'X)^{-1}X'MY $$ Thanks to symmetry of $M$, we have $X'M=X'M'=(MX)'$. Now, it is well-known that $$ MX=(I-X(X'X)^{-1}X')X=X-X(X'X)^{-1}X'X=X-X=0, $$ so that $\hat\delta=0$, too.

The result makes intuitive sense, too: the residuals $\hat\epsilon$ are what you cannot explain by $X$. If you regress that on $X$ again to get $\hat\delta$, a zero effect makes sense.

Christoph Hanck
  • 25,948
  • 3
  • 57
  • 106
  • Thank you, this was exactly what I was looking for , I used unbiasedness for lock of better word to explain my problem. My apologies if it was confusing, again thank you for the answer and for taking your time to write it up! – MisterButter Dec 09 '20 at 13:04
  • If I may, I'm a bit confused by $\hat\delta=(X'X)^{-1}X'MY$. As the OLS formula is $(X'X)^{-1}X'\hat{Y}$. How do you arrive to the conclusion that $\hat\delta=(X'X)^{-1}X'MY$? – MisterButter Dec 09 '20 at 13:11
  • Figured it out, since we have $\hat{\epsilon}=MY$ and not $Y$ in the equality of our model. Dumb question, thanks again! (To much rehearsal and to little attention to learning...) – MisterButter Dec 09 '20 at 13:13
  • @MisterButter, very good. I took the liberty to edit the title of the post to maybe better reflect its content, so please have a look if that is fine with you! – Christoph Hanck Dec 09 '20 at 13:39
  • Thank you very much! – MisterButter Dec 09 '20 at 17:40