9

I've been researching Bayesian linear regression. I'm confused about the following example.

Given the model:

$${\bf y} = {\bf \beta}{\bf X} + \bf{\epsilon} $$

Assuming that ${\bf \epsilon} \sim N(0, \phi I)$ and $p(\beta, \phi) \propto \frac{1}{\phi}$, we have:

$$p(\beta|\phi, {\bf y}) \sim N(({\bf X}^{\text{T}}{\bf X})^{-1}{\bf X}^{\text{T}}{\bf y}, \phi ({\bf X}^{\text{T}}{\bf X})^{-1})$$

How was $p(\beta|\phi, {\bf y})$ determined?

Mateen Ulhaq
  • 115
  • 6
user9171
  • 1,321
  • 3
  • 14
  • 24

1 Answers1

8

Your final formula is missing a left parenthesis.

This is a standard problem that requires no difficult work. The wikipedia page on Bayesian regression solves a harder problem; you should be able to use the same trick (which is basically just a form of completing the square, since you want it in terms of $(\beta - m)' V^{-1} (\beta - m)$ for some $m$ and $V$), with fewer terms to worry about. That is, you get to something like this:

\begin{align}(\mathbf{y}- \mathbf{X} \boldsymbol\beta)^{\rm T}(\mathbf{y}- \mathbf{X} \boldsymbol\beta)&= (\boldsymbol\beta - \hat{\boldsymbol\beta})^{\rm T}(\mathbf{X}^{\rm T}\mathbf{X})(\boldsymbol\beta - \hat{\boldsymbol\beta}) + \mathbf{S} \end{align}

where

\begin{align}\mathbf{S} = (\mathbf{y}- \mathbf{X} \hat{\boldsymbol\beta})^{\rm T}(\mathbf{y}- \mathbf{X} \hat{\boldsymbol\beta})\end{align}

in the exponent.

See also the references at the wikipedia article.

If this is for some subject, please mark it as homework.

Glen_b
  • 257,508
  • 32
  • 553
  • 939