1

to calculate the 95% credible interval in traditional MCMC methods, I take the posterior samples for a parameter and calculate the values where 95% of the probability mass is in. The point estimate is usually the mean.

How is this done in variational inference (VI) methods? From my understanding, VI iteratively improves the point estimates in each step. How can I get the uncertainty associated with the parameters?

spore234
  • 1,323
  • 1
  • 15
  • 31
  • VI will eventually converge, returning the hyperparameters of the parametric variational posteriors. Compute the 95% CI of those posteriors. Incidentally, keep in mind that variational CI may be largely underestimating the true posterior CI (this is because of how VI approximations work). – lacerbi Aug 09 '17 at 13:00
  • can't you draw trials from the posterior distribution of your parameters after the VI? (i am not familiar with this method). Usually what you get with Bayesian inference __is__ the posterior distribution. People use the average or the median due to simplicity (and information theoretical reasons) as far as I am aware. – Guilherme Marthe Aug 09 '17 at 13:02
  • @lacerbi ah that makes sense. I just draw from the posterior estimates, thanks. – spore234 Aug 09 '17 at 13:20
  • 2
    If you're looking for credible intervals, chances are you should prefer MCMC to VI. VI is fast, but biased; MCMC is unbiased, but slow to converge. (See [here].(https://stats.stackexchange.com/questions/271844/variational-inference-versus-mcmc-when-to-choose-one-over-the-other/271862#271862)) More, VI doesn't fit your model of interest: It finds a simplified model nearest in terms of KL divergence. – Sean Easter Aug 11 '17 at 00:43

0 Answers0