0

Currently I am exploring topics for my undergrad thesis. Although I took a course in Bayesian statistics, I am not yet sure how to proceed in finding the posterior in the following case.

I have a d-dimensional prior distribution $\theta \sim \Pi =N(\mu, \Sigma)$. Define $\lambda = sum(\theta)=\theta'\iota$. I have the model distributions $Y|\theta\sim N(\lambda, \tau^2)$. This is clearly a one dimensional random variable, with $\tau$ fixed. How do I compute the posterior distribution $\theta|Y=y$? If I am correct, this should be a multivariate normal distribution again. I would already be really helped by a formula for d=2, or d=3, but for arbitrary d would be amazing.

J. Dekker
  • 108
  • 1
  • 9

1 Answers1

3

Since we seek for a PDF in terms of $\theta$, terms other than $\theta$ can be fused into normalizing constant, which results in the following proportionality argument:

$$p(\theta|Y) = \frac{p(Y|\theta)p(\theta)}{p(Y)}\propto p(Y|\theta)p(\theta)$$

Substituting into RHS yields (let $m=[1,1,...,1]^T\rightarrow \lambda=m^T\theta$): $$\begin{align}p(\theta|Y=y) &\propto \exp\left(-\frac{(y-m^T\theta)^2}{2\tau^2}\right)\exp\left(-\frac{1}{2}(\theta-\mu)^T\Sigma^{-1}(\theta-\mu)\right)\\ &\propto \exp\left(y\frac{m^T\theta}{\tau^2}-\frac{\theta^Tmm^T\theta}{2\tau^2}-\frac{\theta^T\Sigma^{-1}\theta-2\mu^T\Sigma^{-1}\theta}{2}\right) \\ & \propto \exp\left(-\frac{1}{2}\theta^T\left(\Sigma^{-1}+\frac{mm^T}{\tau^2}\right)\theta +\left(\frac{ym^T}{\tau^2}+\mu^T\Sigma^{-1}\right)\theta\right)\end{align}$$

And, this expression is in MV normal form, i.e. it is proportional to $$\exp\left(-\frac{1}{2}(\theta-\mu_{\theta})^T\Sigma_{\theta}^{-1}(\theta-\mu_{\theta})\right)\propto\exp\left(-\frac{1}{2}\theta^T\Sigma_{\theta}^{-1}\theta+\mu_{\theta}\Sigma_{\theta}^{-1}\theta\right)$$ So, we're just going to match term by term and obtain the covariance matrix and mean vector of $\theta$ posterior: $$\Sigma_{\theta}=\left(\Sigma^{-1}+\frac{mm^T}{\tau^2}\right)^{-1}, \mu_{\theta}=\left(\frac{ym^T}{\tau^2}+\mu^T\Sigma^{-1}\right)\Sigma_{\theta}$$

Note: $mm^T$ is actually a matrix of only 1's.

gunes
  • 49,700
  • 3
  • 39
  • 75
  • Thanks a lot! That seems like what I was looking for! I got lost after the first proportionality step, thanks for helping me out! Now I will look at whether this is indeed implementable within the thesis setting that I was looking at! – J. Dekker Apr 08 '19 at 20:09
  • Do you still need help understanding? I'm asking because you accepted the answer. – gunes Apr 09 '19 at 07:56
  • No, I understand it now! Thanks for asking. Turns out my matrix algebra skills were just too rusty after not doing much of that for a year! – J. Dekker Apr 09 '19 at 07:59
  • I just start doubting about the last line.. are you sure that we are only dealing with matrix inverses? I feel that the last Sigma_theta inverse should just be a Sigma_theta? – J. Dekker Apr 18 '19 at 18:53
  • And, the inside should be $\mu^T$ (now changed it) – gunes Apr 18 '19 at 18:57