1

Absolutely stuck on section A of this question I found. I've been trying over 3 hours and still failing, can anyone here guide me through?

Suppose a random sample $x = (x_1, x_2,\ldots, x_n)$ is taken from a normal N($\Theta$, 1) distribution. It is desired to estimate the mean $\Theta$. A normal distribution with zero mean and variance $\frac{1}{t^2}$ is used as a prior distribution for $\Theta$.

(a) Show that the posterior density $\pi(\Theta|x)$ satisfies $$ \pi (\Theta |x)\propto\exp\left \{ -\frac{1}{2}\left ( \Theta ^{2}\left ( n +t^{2} \right ) - 2n\mu \Theta \right ) \right \} $$

where

$$ \mu = \frac{1}{n}\sum_{i=1}^{n}x_{i} $$

By writing this posterior density in the form

$$ \pi (\Theta |x)\propto exp\left ( -\frac{\left ( \Theta -m \right )^{2}}{2\nu ^{2}} \right ) $$

deduce the posterior distribution of $\Theta$.

(b) Using your posterior distribution for $\Theta$, what is your estimate for $\Theta$? By recalling that $X_i - N(\Theta, 1)$, obtain the mean and variance of your estimate. (c) Discuss what happens to your estimate (i) if $n$ is large, (ii) if $t$ is large, (iii) if $t$ is small. (d) Discuss why someone might choose the case (i) large $t$, (ii) small $t$.

$T_{post} = T_{prior} + T_{data}$ but I'm not sure how to utilize this.

For Ci. is it correct that the estimate will become closer to the sample mean when $N$ is large? Cii. make $t$ converge towards the mean? Ciii. the larger the ratio it is towards the estimate?

Avraham
  • 3,182
  • 21
  • 40
Kei Cheung
  • 59
  • 6
  • Are you using the fact that the posterior density is proportional to that quantity? i.e. $$\pi (\Theta |x) \propto \exp\left \{ -\frac{1}{2}\left ( \Theta ^{2}\left ( n +t^{2} \right ) - 2n\mu \Theta \right ) \right \}$$ –  Sep 03 '13 at 20:00
  • Check if this helps: http://stats.stackexchange.com/questions/39261/identifying-a-conditional-posterior-for-mu-and-sigma2/40713#40713 – Zen Sep 04 '13 at 02:02

1 Answers1

1

Solution for part A:

\begin{align*} \pi(\theta|x)&\propto\pi(x|\theta)\pi(\theta)\\ &\propto\exp\left\{-\frac{1}{2}\sum^n_{i=1}(x_i-\theta)^2\right\}\exp\left\{-\frac{t^2}{2}\theta^2 \right\}\\ &\propto\exp\left\{-\frac{1}{2}\sum^n_{i=1}(x_i-\theta)^2-\frac{t^2}{2}\theta^2 \right\}\\ &\propto\exp\left\{-\frac{1}{2}\left[\sum^n_{i=1}(x_i-\theta)^2+t^2\theta^2\right] \right\}\\ &\propto\exp\left\{-\frac{1}{2}\left[\sum^n_{i=1}(x_i^2-2x_i\theta+\theta^2)+t^2\theta^2\right] \right\}\\ &\propto\exp\left\{-\frac{1}{2}\left[-2\theta\sum^n_{i=1}x_i+n\theta^2+t^2\theta^2\right] \right\}\\ &\propto\exp\left\{-\frac{1}{2}\left[-2\theta n\mu+n\theta^2+t^2\theta^2\right] \right\}\\ &\propto\exp\left\{-\frac{1}{2}\left[-2\theta n\mu+\theta^2(n+t^2)\right] \right\}\\ &\propto\exp\left\{-\frac{1}{2}\left[\theta^2(n+t^2)-2\theta n\mu\right] \right\}\\ \end{align*}

So all you have to do is complete the square and you will obtain the kernel for another normal distribution (to complete the problem)

  • er, it's actually suppose to be X with a line on top. I must've typed the wrong thing accidentally, but it wouldn't change anything right? But any tips for the rest of the question? Is Ci etc correct? – Kei Cheung Sep 03 '13 at 20:23
  • 2
    [tag:self-study] questions look for *guidance* instead of detailed answers. You could, e.g., skip from the second to the last line (omitting two mistakes) by noting the argument of $\exp$ is a quadratic function of $\theta$. Its first two coefficients can be read right off: the coefficient of $\theta^2$ is evidently $-n/2$ from the sum plus $-t^2/2$ and the coefficient of $\theta$ can be found by letting $\theta=0$ in the derivative $\sum_i(x_i-\theta)-t^2\theta$, giving $n\mu$ (*not* $\mu$). The constant coefficient (not involving $\theta$) will be absorbed in the constant of proportionality. – whuber Sep 03 '13 at 20:48
  • whuber, is there anything you can guide me to read up. I have troubles following your explanation such as "like he coefficient of θ2 is evidently −n/2 from the sum plus −t2/2". – Kei Cheung Sep 03 '13 at 21:00
  • 1
    I hate to say it, but this is elementary algebra: you just look at where $\theta$ can be multiplied by $\theta$ and add up all the coefficients. By examining $(x_i-\theta)^2$ you know the only way to obtain $\theta^2$ comes from multiplying $-\theta$ by $-\theta$, giving a coefficient of $1$; there are $n$ of these in the sum; and that sum is multiplied by $-1/2$: altogether that gives a coefficient of $-n/2$. To this is added the coefficient $-t^2/2$ that explicitly appears. (The derivative trick is elementary too (it doesn't really use Calculus) but is not usually taught in high schools.) – whuber Sep 03 '13 at 22:41
  • I edited my arithmetic mistake. –  Sep 03 '13 at 22:48
  • oh right, yeah sorry. now that you expanded on it, it kinda hit me on the face. Still a bit of trouble finding resources to help me with the other parts though. Trying to get back into this is much more difficult than I thought – Kei Cheung Sep 03 '13 at 23:06