0

I am trying do inference on a problem with a posterior distribution with two parameters (not in closed form).

I have built a grid approximation and have been able to create a contour plot of this posterior distribution around reasonable values for the 2 parameters (computed with linear regression). I am also able to generate samples out of this distribution.

I would like to approximate this contour with a normal (from BDA):

$$ p(\theta|y) \approx N(\hat{\theta}, [I(\hat{\theta})]^{-1}) $$

where

$$ I(\theta) = -\frac{d^2}{d\theta^2}\log p(\theta|y) $$

Now, I don't really know how to compute $I(\hat{\theta})$ from my $\theta_{1..n}$ samples. I guess I can use the mean of my samples as the mean for the normal approximation, but I don't know how to compute the variance.

Any ideas how to do this?

JC1
  • 217
  • 1
  • 8
  • If you can compute the mean of your samples, then why can't you also compute their covariance? (Note that your question does not really have to do with the central limit theorem.) Now whether or not a normal approximation is appropriate, that is another question. (For example if your distribution is multi-modal, then you may want to approximate each mode as a separate Gaussian.) – GeoMatt22 Oct 04 '16 at 01:59
  • Yes, I was missing the relationship between the Fisher Information Matrix $I(\hat{\theta})$ and the covariance matrix. This question http://stats.stackexchange.com/questions/68080/basic-question-about-fisher-information-matrix-and-relationship-to-hessian-and-s clarified that. – JC1 Oct 04 '16 at 05:11

0 Answers0