1

I am estimating a model using MCMC (Gibbs Sampling). Because of the complexity of the model, I have been running two chains with many iterations.

A plot of the draws for each parameter reveals a spike in the variance of the draws. After an additional few thousand iterations, the plot returns to the region of the distribution where has spent most of its time. I have run two chains in parallel with different starting values and different seeds for the random number generator. One chain does this at around 50,000th iteration, and the other does this around 25,000th iteration.

Here is a plot of a parameter sampled from an inverse gamma posterior in the first chain:

Posterior Sample from Inv. Gamma Distribution

Here are similar results - but appropriately more symmetric - for a different parameter sampled from a normal posterior from the same chain:

Posterior Sample from Normal Distribution

The entire support of these distributions will be covered over an infinite number of iterations, but these spikes seem odd for a finite number of samples.

I have looked through my code to determine what could be going wrong and not found any apparent issues. I have seen a similar question asked here, but there hasn't been much discussion.

Any feedback on this behavior would be appreciated!

Alex
  • 37
  • 5
  • What do you mean "inverse gamma posterior" and "normal posterior"? Do you mean "inverse gamma full conditional" instead? Also, it would be good to share some code here, because there is not enough information about the model or the sampler to say anything with certainty. – Greenparker Mar 02 '18 at 15:31
  • What are your priors on the variable in question? – jbowman Mar 02 '18 at 15:37
  • Yes, @Greenparker, I do mean the full conditional. I would like to share some code here, but there's too much to share. – Alex Mar 02 '18 at 15:48
  • @jbowman, this is a regression model with the noninformative prior proportional to sigma^-2, where sigma is the standard deviation of the residuals. The likelihood is normal. I will update the original post. – Alex Mar 02 '18 at 15:50
  • You might want to use a more-informative weakly informative prior to prevent this sort of thing from happening; see the answers to this question: https://stats.stackexchange.com/questions/6493/weakly-informative-prior-distributions-for-scale-parameters and the excellent Gelman paper pointed to at the link. – jbowman Mar 02 '18 at 15:53
  • @jbowman, that's a good idea. I will see what that comes up with. – Alex Mar 02 '18 at 16:06
  • When huge spikes occur and an improper prior like $\pi(\sigma)=\sigma^{-2}$ is used, there is always the "usual suspect" beyond coding errors, namely the possibility that the posterior is not defined although the full conditional posteriors are. This happened in an early paper of Gelfand et al. (1990). – Xi'an Mar 02 '18 at 16:22

0 Answers0