0

We can assume all the assumptions of the classical setting of linear regression with unknown variance of the residual.

What would be the multivariate distribution of the vector $\hat{\beta}$?

The result should be consistent with the fact that the marginal distribution of each element of $\hat{\beta}$ is a t-distribution.

We should also be able to compute the distribution of any linear combination of $\hat{\beta}$.

denizen of the north
  • 848
  • 1
  • 8
  • 15
  • The marginal distributions are *Normal,* not Student t, when the errors are Normal. This is a FAQ, so you can discover the answer by searching our site for questions about the covariance of multiple regression estimates. – whuber Nov 20 '19 at 18:24
  • @whuber when is it student-t then? – denizen of the north Nov 20 '19 at 18:25
  • One would have to conceive of a multivariate distribution of errors which, when linearly transformed according to the OLS formula, has a multivariate T distribution. Herein lies the crux of the matter: the parameter estimates are a linear function of the responses. This makes it fairly straightforward to analyze the distribution of parameter estimates based on any given distributional assumptions about the error terms. – whuber Nov 20 '19 at 18:28
  • Sure. I suppose what you are saying is that if x is fixed and y is normal, then $(X^tX)^{-1}X^ty$ is a linear transformation of a multivariate normal, so it is also normal. But there is also the problem estimating the variance of $c\hat{\beta}$ using the sample... – denizen of the north Nov 20 '19 at 18:39
  • That's answered directly by the first duplicate. – whuber Nov 20 '19 at 18:40
  • @whuber Going through the answer...i suppose the numerator is normal and the denominator $y^t(I-H)y$ is chi-square with a degree of freedom equal to the rank(X)? So that produces a t-distribution? – denizen of the north Nov 20 '19 at 18:59
  • Yes, that's right. But let's be clear: you are now referring to the t statistics for testing hypotheses about the parameters, rather than about the distributions of the parameter estimates themselves. – whuber Nov 20 '19 at 19:29

1 Answers1

0

Likelihood theory tell us that

$$\hat{\beta} \sim \mathcal{N}(\beta, \Sigma) $$

Where $\Sigma = \hat{\sigma}(X'X)^{-1}$. The t-distribution comes from the fact that the variance of the betas includes the estimate of the noise, $\hat{\sigma}$. So when you compute your test statistic for the betas, you get a normal random variable divided by the root of chi-square random variable divided by it's degrees of freedom, which is a t random variable.

Demetri Pananos
  • 24,380
  • 1
  • 36
  • 94