2

I recently came across this question on stats stack exchange where a user asked for the proof that the T statistic has a t distribution. The question assumes a linear model of the form;

$$Y=X\beta+\epsilon$$

The proof in the accepted answer makes a step in which it first defines the residual maker matrix as:

$$M=I_n - X(X^TX)^{-1}X^T$$

And then states $My=\hat{\varepsilon}$ (the estimated residuals) and that $M$ is symmetric and idempotent. I understand this so far. Later on though it makes this step:

$$\frac{\hat\varepsilon^T\hat\varepsilon}{\sigma^2} = \left(\frac{\varepsilon}{\sigma}\right)^T M \left(\frac{\varepsilon}{\sigma}\right)$$

I am confused though as, given $My=\hat{\varepsilon}$, it seems this should equal:

$$\frac{\hat\varepsilon^T\hat\varepsilon}{\sigma^2} = \left(\frac{y}{\sigma}\right)^T M \left(\frac{y}{\sigma}\right)$$

However, if you were to change this the original proof no longer seems to work. Have I misunderstood this?


Note: I have also asked this question as a comment on the original post but it seems as though the author of the solution might not have logged in for a long time so made this separate question. I will add a comment to the original post if any answers are added here.

JDoe2
  • 576
  • 3
  • 14

1 Answers1

2

First you replace $y=M\hat\varepsilon$:

$$\frac{\hat\varepsilon^T\hat\varepsilon}{\sigma^2}=\frac{(My)^T\cdot(My)}{\sigma^2}=\frac{y^TM^TMy}{\sigma^2}$$

Now, remember that $M$ is symmetric and idempotent:

$$\frac{y^TM^TMy}{\sigma^2}=\frac{y^TM^2y}{\sigma^2}=\frac{y^TMy}{\sigma^2}$$

Now you have $\frac{\hat\varepsilon^T\hat\varepsilon}{\sigma^2}=\left(\frac{y}{\sigma}\right)^TM\left(\frac{y}{\sigma}\right)$, as expected!

And observing that $My=M\epsilon$ the result

$$\frac{\hat\varepsilon^T\hat\varepsilon}{\sigma^2}=\left(\frac{\epsilon}{\sigma}\right)^TM\left(\frac{\epsilon}{\sigma}\right),$$ follows.

It is easy to see that $My=M\epsilon$ simply by using the model equation in matrix form $y = X\beta + \epsilon$ and then premultiplying with $M$ to get

$$My= MX\beta + M\epsilon = M\epsilon,$$

where it is used that $MX = (I - X(X^\top X)^{-1}X^\top )X = X - X = \boldsymbol 0$

Jesper for President
  • 5,049
  • 1
  • 18
  • 41
PedroSebe
  • 2,526
  • 6
  • 14