2

In the case of simple linear regression, I understand the math behind the variance of the estimates:

$$ \operatorname{Var}(\widehat{\beta}_0) = s^2 \bigg(\frac{1}{n} + \frac{\bar{x}^2}{S_{xx}} \bigg) \hspace{12pt} \text{and} \hspace{12pt} \operatorname{Var}(\widehat{\beta}_1) = \frac{s^2}{S_{xx}} $$

However, I don't follow the steps that lead us to conclude that

$$ \operatorname{Var}(\widehat{\beta}_0 + \widehat{\beta}_1x_*) = s^2 \bigg(\frac{1}{n} + \frac{(x_* - \bar{x})^2}{S_{xx}} \bigg)$$

I can see that this resembles a sum of the variances of the parameter estimates, but I don't see how we get the $(x_* - \bar{x})^2$ term.

N4v
  • 155
  • 5
  • 2
    That $s^2$ you have is usually denoted by $\sigma^2$, variance of the errors. And did you use $\mathrm{Var}(\hat{\beta_0} + \hat{\beta_1}x_*) =\mathrm{Var}(\hat\beta_0)+x_*^2\mathrm{Var}(\hat\beta_1)+2x_*\mathrm{Cov}(\hat\beta_0,\hat\beta_1)$? – StubbornAtom Dec 30 '18 at 15:07
  • I considered that, but isn't the covariance between the two parameter estimates 0? – N4v Dec 30 '18 at 15:46
  • 1
    Why zero? Not at all in general. You have to find the covariance. – StubbornAtom Dec 30 '18 at 16:37
  • 2
    @N4v : \mathrm{} and \operatorname{} do not always yield identical results. $$ \begin{align} & \text{a \mathrm{Var} X} \quad & & a\mathrm{Var} X \\ \\ & \text{a\operatorname{Var} X} \quad & & a\operatorname{Var} X \end{align} $$ The second version is standard. With \operatorname{}, the spacing depends on the context, so that with $\operatorname{Var}X$ you see more space after "Var" than with $\operatorname{Var}(X). \qquad$ – Michael Hardy Dec 30 '18 at 16:51
  • 1
    N4v, you're almost right: the parameter estimates are uncorrelated when the regressors are mean-centered. – whuber Dec 30 '18 at 17:04

1 Answers1

2

$$ \operatorname{var}\left(\widehat\beta_0 + \widehat\beta_1 x_* \right) = \operatorname{var}\left( \widehat\beta_0 \right) + x_*^2 \operatorname{var}\left( \widehat\beta_1 \right) + {} \underbrace{2 x_* \operatorname{cov}\left( \widehat\beta_0, \widehat\beta_1 \right)}. $$ The posted question does not mention the term over the $\underbrace{\text{underbrace}}.$

This is simpler if you write $$ \operatorname{var}\left( \overline y + \widehat\beta_1(x_*-\overline x) \right) = \operatorname{var}\left( \,\overline y\,\right) + (x_* - \overline x)^2 \operatorname{var}\left( \widehat\beta_1 \right) $$ In that case, the covariance is $0.$

Michael Hardy
  • 7,094
  • 1
  • 20
  • 38
  • Thanks, but why are the parameter estimates uncorrelated when they are mean-centered? Does this come from the covariance matrix $\sigma^2 (X^{T}X)^{-1}$, where $X$ is the design matrix? – N4v Dec 30 '18 at 21:02
  • 1
    @N4v : Yes. When the $x$-values are mean centered, then the two columns of the design matrix are orthogonal to each other. – Michael Hardy Dec 31 '18 at 05:46
  • 1
    @N4v See https://stats.stackexchange.com/questions/68151/how-to-derive-variance-covariance-matrix-of-coefficients-in-linear-regression?noredirect=1&lq=1 , https://stats.stackexchange.com/questions/104704/are-estimates-of-regression-coefficients-uncorrelated, https://math.stackexchange.com/questions/2902663/how-to-show-operatornamecovb-0-b-1-frac-sigma-barxs-xx?rq=1 for the covariance between the least square estimates. – StubbornAtom Dec 31 '18 at 11:54