4

I am having problems calculating the variance of a residual in an SLR setting,
ie $\text{var}$$(y_i- \hat{y_i})$. Here is what I have thus far.

If $ \hat{y_i}= \hat{\beta_0} + \hat{\beta_1}x_i$ and

$ \ y_i= \beta_0 + \beta_1x_i$

(where $\hat{\beta_0}$ and $\hat{\beta_1}$ are ordinary least squares estimates of $\beta_0$ and $\beta_1$).

$ \text{var}$$(e_i)=\text{var}$$(y_i- \hat{y_i})$
$ \ =\text{var}$$(y_i)+\text{var}$$(\hat{y_i})-2*\text{cov}$$(y_i, \hat{y_i}) $
$ \ =\sigma^2 + \sigma^2( \frac{1}{\ n} +\frac{(x_i-\bar{x})^2\sigma^2}{\sum(xi-\bar{x})^2})-? $

Now what is screwing me up is the covariance term. When I was studying prediction intervals we assume that $ \hat{y_i} $ has no bearing on $\ y_i $. I was told by classmate that this is not the case and that $ \text{cov}$$(y_i, \hat{y_i})=\text{var}$$(y_i) $. Which option is correct, if any? And what is the difference between the variance we find for prediction intervals and this one?

Thanks in advance!

Alucardex
  • 43
  • 3

1 Answers1

1

When doing prediction intervals, you're doing that calculation for an observation that's not used in the estimation, so (by the regression assumptions themselves) $\hat{y}_i$ has no bearing on $y_i$.

With residuals the observation is used in the estimation, so the two are dependent.

$\text{cov}(y_i, \hat{y}_i) = \text{cov}(\hat{y}_i+y_i-\hat{y}_i, \hat{y}_i) = \text{cov}(\hat{y}_i, \hat{y}_i) +\text{cov}(y_i-\hat{y_i}, \hat{y}_i) = \text{var}(\hat{y}_i)+0$

For additional details, see here

Community
  • 1
Glen_b
  • 257,508
  • 32
  • 553
  • 939
  • Would it be correct to say that instead of the covariance of a specific $y_i$ and $\hat{y}_i$, if I take that of $y$ and $\hat{y}$, then I would get: $\operatorname{Cov}(y, \hat{y})=\sum(y_i - \bar{y})(\hat{y}_i - \bar{\hat{y}}_i)/(n-1)=\sum(y_i - \bar{y})(\hat{\beta}_1 x_i+\hat{\beta}_0 - \hat{\beta}_1\bar{x}-\hat{\beta}_0)/(n-1)=\sum(y_i - \bar{y})(\hat{\beta}_1(x_i-\bar{x}))/(n-1)=\hat{\beta}_1\sum (y_i-\bar{y})(x_i - \bar{x})/(n-1)=\hat{\beta}_1\operatorname{Cov}(x,y)$ – user1879926 Apr 10 '20 at 06:04