2

For a simulation study I derived the posterior distribution of a parameter $\theta$ with data $D$ as $p(\theta|D)$. I can sample from the posterior, but there is no analytic expression. The posterior allows me to calculate the 95% credible intervals as the 2.5 and 97.5 percentile of a sampled Monte Carlo distribution. That is, I draw repeatedly from the posterior and then read off the quantiles.

I want to know whether the credible interval I calculated really has a 95% probability of covering the true value. The true value of $\theta$ is known as this is a simulation. So I sampled repeatedly (1000 times) data $D$ from a superpopulation and calculated the credible interval each time. It turned out that the credible interval only covered the true parameter in about 90% of the cases.

From here and here I believe I understand that the credible interval should indeed have 95% coverage of $\theta$ but in my case it does not have. Did I evaluate the coverage of my credible interval correctly? What are possible reasons why in my monte carlo simulation the coverage is not 95%?

tomka
  • 5,874
  • 3
  • 30
  • 71
  • 2
    As pointed out in the second post you link to, you need to first simulate $\theta$ from the prior (so it needs to be proper) and then simulate the data given $\theta$ and finally compute the credible interval based on the same prior. Then the nominal coverage is guaranteed if your code is correct. If you use a constant $\theta$ then the posterior will be biased towards the prior so it's not surprising that the coverage then is lower. – Jarle Tufto Sep 14 '17 at 10:07

0 Answers0