1

In a linear regression model where $Y = X\beta+\epsilon$, with a residual vector $e=y-X\hat\beta$ where $\hat\beta=(X^TX)^{-1}X^Ty$ is the optimal regression coefficients given the data $X$ and $y$. Here $Var(e)=Var((I-X(X^TX)^{-1}X^T)y)=\sigma^2(I-H)$ if $H= X(X^TX)^{-1}X^T$ and $Var(\epsilon) = \sigma^2I$, based on equation 10 on the second page here. To me, $Var(e_i) = \sigma^2(I-H)_{ii}$ which is generally not constant for all $i$.

However, the same author here states that $Var(e_i)=\frac{n-2}{n}\sigma^2$ (page 3) and states that the residuals should show constant variance, unchanging with $x$ (page 5). On page 7 he further states that a plot of the residuals against the predictors should be look like a constant-width of points around a flat line of height zero since variance of the residual is constant and has expected value 0.

Is $Var(e_i)$ constant? Why do the two formulations above appear to say different things?

Yandle
  • 743
  • 2
  • 12
  • Answered here: https://stats.stackexchange.com/questions/291780/in-simple-linear-regression-how-does-the-derivation-of-the-variance-of-the-resi – kjetil b halvorsen Apr 17 '20 at 05:40
  • @kjetilbhalvorsen I saw that post as well, however if that's true then I don't get what my second source was getting at when it states explicitly that the variance of the residuals are constant with x – Yandle Apr 17 '20 at 14:37
  • Note that he continues "though again I omit the details, so as not to spoil a future assignment". He just botched it. "Omitting the details" (usually from laziness) is where I am introducing all kinds of errors, at least. But just DO that exercise, and you will see he is wrong! – kjetil b halvorsen Apr 17 '20 at 19:29
  • Does this answer your question? [In simple linear regression, how does the derivation of the variance of the residues support its 'Constant Variance' Assumption?](https://stats.stackexchange.com/questions/291780/in-simple-linear-regression-how-does-the-derivation-of-the-variance-of-the-resi) – kjetil b halvorsen Apr 17 '20 at 19:32

0 Answers0