0

Let $Y_{i} = \beta_{0} + \beta_{1}x_{i} + \epsilon_{i}$ $(i = 1,2,\ldots,n)$, where $\textbf{E}[\epsilon] = 0$ and $\textbf{Var}[\epsilon] = \sigma^{2}\textbf{I}_{n}$. Find the least square estimates of $\beta_{0}$ and $\beta_{1}$. Prove they are uncorrelated if and only if $\overline{x} = 0$.

MY ATTEMPT

As far as I have understood, $(\hat{\beta}_{0},\hat{\beta}_{1})^{\prime} = (\textbf{X}^{\prime}\textbf{X})^{-1}\textbf{X}^{\prime}\textbf{Y}$, where $\textbf{X}$ is the regression matrix. But I do not know how to answer the second question. Can someone help me out? Thanks in advance!

EDIT

The variance of $(\hat{\beta}_{0},\hat{\beta}_{1})$ is given by \begin{align*} \textbf{Var}((\textbf{X}^{\prime}\textbf{X})^{-1}\textbf{X}^{\prime}\textbf{Y}) = (\textbf{X}^{\prime}\textbf{X})^{-1}\textbf{X}^{\prime}\textbf{Var}(\textbf{Y})\textbf{X}(\textbf{X}^{\prime}\textbf{X})^{-1} = \sigma^{2}(\textbf{X}^{\prime}\textbf{X})^{-1} \end{align*}

given that $\textbf{Var}(\textbf{Y}) = \textbf{Var}(\textbf{Y} - \textbf{X}\beta) = \textbf{Var}(\epsilon) = \sigma^{2}\textbf{I}_{n}$. Consequently, we have \begin{align*} \textbf{X}^{\prime}\textbf{X} = \left[ {\begin{array}{cc} n & \displaystyle\sum_{k=1}^{n}x_{k}\\ \displaystyle\sum_{k=1}^{n}x_{k} & \displaystyle\sum_{k=1}^{n}x^{2}_{k} \\ \end{array} } \right] \Longrightarrow (\textbf{X}^{\prime}\textbf{X})^{-1} = \frac{1}{\det(\textbf{X}^{\prime}\textbf{X})}\left[ {\begin{array}{cc} \displaystyle\sum_{k=1}^{n}x^{2}_{k} & -\displaystyle\sum_{k=1}^{n}x_{k}\\ -\displaystyle\sum_{k=1}^{n}x_{k} & n \\ \end{array} } \right] \end{align*}

Therefore $\text{Cov}(\hat{\beta}_{0},\hat{\beta}_{1}) = 0$ iff $\displaystyle\sum_{k=1}^{n}x_{k} = 0$, that is, $\overline{x} = 0$. Am I on the right track?

  • Can you find the covariance between $\hat\beta_0$ and $\hat\beta_1$, the least square estimates? – StubbornAtom Mar 31 '19 at 19:44
  • I have edited my answer. Can you tell me if I am on the right track? –  Mar 31 '19 at 20:10
  • See https://stats.stackexchange.com/questions/104704/are-estimates-of-regression-coefficients-uncorrelated, https://stats.stackexchange.com/questions/123481/are-the-estimates-of-the-intercept-and-slope-in-simple-linear-regression-indepen. – StubbornAtom Mar 31 '19 at 20:42
  • I find it interesting to contemplate the case where all the $x_i$ are equal to a common nonzero value, because this would seem to provide a counterexample, depending on what one means by "least squares estimates" in this case. – whuber Mar 31 '19 at 20:55
  • In this case, I am assuming $n\geq p$ and $\textbf{X}$ has full rank. Otherwise, I think it would be necessary to make use of pseudo inverse, but I have not studied it so far. –  Mar 31 '19 at 21:23
  • Thank you. In that case, there seems to be nothing left to say: you have answered your question within the edit. – whuber Apr 03 '19 at 16:49

0 Answers0