1

How do I derive prediction intervals for a general linear model?

My general linear model written in matrix form is,

$$ \mathbf{Y} = \mathbf{X} \mathbf{B} + \mathbf{R}$$

with each of the rows of the residual matrix $\mathbf{R}$, distributed as $\mathbf{r}_i \sim \mathcal{N}(\mathbf{0},\Sigma_{R})$. Alternatively, $row(\mathbf{R}) \sim \mathcal{N}(\mathbf{0}, \mathbf{I} \otimes \Sigma_{R})$.

I have a least squares estimator as

$$\hat{\mathbf{B}} = (\mathbf{X}^{T} \mathbf{X})^{-1} \mathbf{X}^{T} \,, $$

from affine transformations on $\mathbf{R}$, this estimator is distributed as,

$$ row(\hat{\mathbf{B}}) \sim \mathcal{N}(\mathbf{B}, \Sigma_{\hat{\mathbf{B}}}) $$

What I wish to do is form a prediction interval on a new instance of $\mathbf{Y}$ that is within the range (an interpolation, not extrapolation). This new instance is $\tilde{\mathbf{y}}_{i}$. What I wish to do is find a prediction interval (or confidence interval?) on this new instance.

On my own, I have used affine transformations again on $\hat{\mathbf{B}}$ to find that

$$\tilde{\mathbf{y}}_{i} \sim \mathcal{N}(\tilde{\mathbf{x}} \hat{\mathbf{B}}, \mathbf{X}^{T} (\mathbf{X}^{T} \mathbf{X})^{-1} \mathbf{X} \otimes \hat{\Sigma}_{R} ) $$

However I am uncertain in my solution. My textbooks on regular multiple linear regression have formulas for prediction intervals. But if the LS estimator has a distribution, why not just use this result? From it I can make $1\sigma$ or $3 \sigma$ intervals etc. In short,

$\textbf{How does one derive prediction intervals for the general linear model? }$

bill_e
  • 2,681
  • 1
  • 19
  • 33
  • The formula for iid errors is given at http://stats.stackexchange.com/questions/9131. Although your question refers to the "general" linear model, the estimator you offer is *not* the one given by the GLM--it is the OLS estimator which is appropriate only for constant diagonal $\Sigma_R$. This calls into question what you are really looking for. Could you make this a little clearer? – whuber Feb 19 '14 at 21:27
  • $row(\mathbf{R})$ is block diagonal, while $\Sigma_{R}$ is not. OLS is appropriate since each vector $\mathbf{r}_{i}$ is iid distributed... right? – bill_e Feb 19 '14 at 22:03
  • It was not originally evident that you have assumed $Y$ is a *vector* response. Your edit helps clarify that, thank you (although to me "general linear model" implies you are contemplating the possibility of an arbitrary covariance structure for the residuals, not just iid residuals). It sounds now like you are looking for the multivariate generalization of the formula Rob Hyndman gave in the thread I referenced, right? – whuber Feb 19 '14 at 22:10
  • 1
    Sorry my question was unclear, thank you for following up! Yes, the multivariate version of what you linked is what I'm after. Would also appreciate a nudge on how to derive it for myself, not just the formula (that I just can't seem to find). Thank you! – bill_e Feb 19 '14 at 22:12
  • 1
    Well, if I had to derive a formula I might begin with the generalized least squares version of the univariate case, because the multivariate model can be put into that form (as you indicate after "alternatively"). – whuber Feb 19 '14 at 22:18
  • Thats true. After doing that I obtain the formula I get at the end of my Q. From there, how to I get to either a prediction/confidence interval? – bill_e Feb 19 '14 at 22:20
  • 1
    This stuff can get confusing. When I recently encountered such a model, and had to explain it to non-statistical people, I found it useful to [spell out the details explicitly for a small example](http://gis.stackexchange.com/questions/84449/what-is-the-process-to-calculate-helmert-parameters/84454#84454). – whuber Feb 19 '14 at 22:24

0 Answers0