0

In the linear model $\mathbf{Y} = \mathbf{X}\beta + \epsilon$, where $\epsilon \sim N(0, \sigma^2 \mathbb{I})$, it is known the the standard error of the estimator $\hat{\beta}$ is given by

$$Var(\hat{\beta})=\sigma^2(\mathbf{X}^T\mathbf{X})^{-1}. $$

In one of the posts (How are the standard errors of coefficients calculated in a regression?), a representation was made as follows:

$$ \mathbf{X}= \begin{pmatrix}1 & x_1 \\ 1 & x_2 \\ \vdots & \vdots \end{pmatrix} \qquad \text{and} \qquad \beta = \begin{pmatrix} a \\ b \\ \vdots \end{pmatrix}$$

Question: Is $\mathbf{X}$ here just a simplifying assumption? If I multiply $\mathbf{X}\beta$ (plus the error term), then it becomes $y_i = a + bx_i + \epsilon_i$, which is nicely the linear regression model.

Question: With the result of $Var(\hat{\beta})$ above, how can I get the standard error for the slope and intercept estimates, $Var(\hat{\beta_0})$ and $Var(\hat{\beta_1})$? (Assuming $Y={\beta_0}+\beta_1X $).

Edit: In the link I gave, I was able to follow the derivation. I just didn't understand:

$$\sqrt{\widehat{\textrm{Var}}(\hat{b})} = \sqrt{[\hat{\sigma}^2 (\mathbf{X}^{\prime} \mathbf{X})^{-1}]_{22}} = \sqrt{\frac{n \hat{\sigma}^2}{n\sum x_i^2 - (\sum x_i)^2}}.$$

What is $[\cdots]_{22}$ inside the square root?

Your insights would be great.

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
cgo
  • 7,445
  • 10
  • 42
  • 61
  • 2
    You didn't successfully reproduce what was at the post, nor did you quite generalize it. If your $X$ only has 2 columns, $\beta$ can only have two elements (as at the linked post), and you'll be doing simple linear regression. Generally in multiple regression, $X$ will have $p+1$ columns (including the constant) and $\beta$ will have $p+1$ elements. – Glen_b Apr 10 '15 at 05:50
  • The post you linked to itself links to discussion of the [general case](https://en.wikipedia.org/wiki/Linear_regression). – Glen_b Apr 10 '15 at 05:55
  • 1
    $[]_{22}$ means the entry in the 2nd row and 2nd column of the matrix between the square brackets, i.e. the variance of $\hat{b}$ in this case. – ocram Apr 10 '15 at 06:00
  • Is there a reason why we take the second row, second column? (I am not familiar statistics using matrices.) – cgo Apr 10 '15 at 06:04
  • 1
    In a variance-covariance matrix of a vector, the variances are on the diagonal and covariances are off-diagonal. You want the variance of the second component of the vector, which will be the second element along the diagonal of the variance-covariance matrix, hence the 2,2. – Silverfish Apr 10 '15 at 06:41
  • Does it mean that if I want to take the variance of the first term, then I do the same, except take the first term and first column + variance of the error term? – cgo Apr 10 '15 at 07:10

0 Answers0