Bayesian Estimation: Bernoulli and Quadratic Loss Function

Question

I am trying to understand a solution to this problem (I am a very beginner in Bayesian statistics) and I am terribly confused so I would appreciate it if someone could explain to me how exactly this risk function was obtained. I would also appreciate any pointers/advice on the literature where I can encounter similar problems and a good explanation of the core concepts: $$ X_{1}, . . . X_{n} $$ is Bernoulli with unknown parameter $ \theta_{0} $

$$ \hat\theta_{1} = \bar X $$

$$\hat\theta_{2}=\dfrac{n\bar X + a}{n + c}$$
and $ a<c $

The risk for $ \hat\theta_{1} $ is $$R\left(\hat\theta_{1},\theta_{0}\right) = \frac{1}{n}\theta_{0}(1 - \theta_{0}) $$

The risk for $\hat\theta_{2} $ is $$R\left(\hat\theta_{2},\theta_{0}\right) = \frac{1}{(c+n)^2}[(a -\theta_{0}c)^2+n\theta_{0}(1-\theta_{0})] $$

So my problem is, I think I understand how the bias was derived but I do not really understand why the variance is multiplied by n , i.e. $n\theta_{0}(1-\theta_{0})$? Actually, when I square the Bias, I do not understand what happens to $ n{E}[\bar X] $ and $ - \theta_{0} $ when I plug $\hat\theta_{2}$ in $ ({E}(\hat\theta) - \theta_{0})^2 $ . Are they equal? If so, why are they equal?

And I am also confused by this result for $$ \hat\theta_{1} = \frac{n\bar{X}+\sqrt{n}/2}{n + \sqrt{n}} $$ which corresponds to $$ a= \sqrt{n}/2 \text{ and } c = \sqrt{n} $$ and which has a risk equal to $$ R(\hat\theta_{1},\theta_0) = \frac{1}{4n} \frac{n^2}{(n + \sqrt{n})^2} $$

When I plug in these values in $R\left(\hat\theta_{2},\theta_{0}\right)$, I do not understand where $n^2$ in the numerator and $4n$ in the denominator come from.

Thank you in advance for any advice/recommendations.

Answer 1

The risk for $\hat\theta_{2} $ is \begin{align*} R(\hat\theta_{2},\theta_{0}) &= \mathbb{E}_{\theta_0}\left[ (\hat\theta_2(X)-\theta_0)^2 \right] \\ &= \mathbb{E}_{\theta_0}\left[ \left(\frac{n\bar{X}+a}{n+c} - \theta_0\right)^2 \right] \\ &= \left(\frac{n\mathbb{E}_{\theta_0}[\bar{X}]+a}{n+c} - \theta_0\right)^2 + \text{var}_{\theta_0}\left(\frac{n}{(n+c)}\,\bar{X}\right)\\ &= \left(\frac{n\theta_0+a}{n+c} - \theta_0\right)^2 + \frac{n^2}{(n+c)^2}\,\text{var}_{\theta_0}(\bar{X})\\ &= \left(\frac{a-c\theta_0}{n+c}\right)^2 + \frac{n^2}{(n+c)^2}\,\frac{\theta_0(1-\theta_0)}{n}\\ &= \frac{1}{(c+n)^2}[(a -\theta_{0}c)^2+n\theta_{0}(1-\theta_{0})] \end{align*}

So you get the squared bias plus the variance of the Bayes estimator, which is the variance of the sample average multiplied by the square of the coefficient.

The second part follows from using $$ a= \frac{1}{2} \sqrt{n} \ \text{ and }\ c = \sqrt{n} $$ in the above general formula and expanding the two squared quantities: \begin{align*} \frac{1}{(\sqrt{n}+n)^2}&[\{(1/2)\sqrt{n} -\theta_{0}\sqrt{n}\}^2+n\theta_{0}(1-\theta_{0})] \\ &= \frac{n}{(\sqrt{n}+n)^2}\left[ \frac{1}{4} -\theta_0+\theta_0^2+\theta_0-\theta_0^2 \right]\\ &= \frac{1}{4(1+\sqrt{n})^2} \end{align*}