0

What are the main differences between approximate bayesian computation vs approximate bayesian inference?

Are they essentially the same?

Do they refer to the same of different family of models?

My initial understanding was that bayesian computataions refer to approaches that are used when the likelihood or analytic form of the formulation is intractable and that bayesian inference was for methods when the posterior is intractable?

Am I thinking this wrong?

Kirk Walla
  • 102
  • 7
  • Do you mean " that Bayesian inference was for methods when the posterior is _tractable_"? – Xi'an Nov 18 '19 at 05:42

1 Answers1

1

Approximate Bayesian Computation (ABC) is a family of methods that can be used when likelihood is intractable.

Approximate Bayesian Inference seems to be used ambiguously, either as a synonym for ABC, or for Variational Inference, i.e. class of methods that given (trackable) likelihood and priors approximate the posterior distribution.

Neither of those have anything to do with having closed-form solution for posterior. If you don't, you can simply use Markov Chain Monte Carlo, or other approach to sampling from, or estimating the posterior.

Tim
  • 108,699
  • 20
  • 212
  • 390
  • Thanks for the reply. I was under the impression that MCMC, VI & Laplace approximations belong to Approx. Bayesian Inference? Also ABC can be used when the density function is intractable? – Kirk Walla Nov 17 '19 at 22:42
  • 1
    @kirk I don't recall it being used like this. This would mean that Bayesian inference is approximate Bayesian inference, since there's almost nothing left beyond this. – Tim Nov 18 '19 at 05:39
  • 1
    Contrary to ABC, VI, Laplace or EP, MCMC (like Monte Carlo) is not an approximate method in the sense that increasing the computing time increases the precision of the result. – Xi'an Nov 18 '19 at 05:41
  • @Xi'an & Tim, thank you folks for the help. My confusion is based on reading a thesis and a paper. The thesis talks about ABC and specifically that they're used when either the likelihood or density func. are intractable. On the other hand the paper has the following quote: "The posterior is intractable. 3 standard methods of approx. Bayes infer. exist: MCMC, VI, Laplace approx.". Is this statement wrong? – Kirk Walla Nov 18 '19 at 09:36
  • @kirk maybe you could share the source of the quote for some context? – Tim Nov 18 '19 at 10:15
  • Sure thing, [paper](http://www.gatsby.ucl.ac.uk/~balaji/udl-camera-ready/UDL-15.pdf). Look at introduction, second paragraph. – Kirk Walla Nov 18 '19 at 11:00
  • @Tim, let me know what you think? Am I reading this wrong or is it wrongly stated by the authors? – Kirk Walla Nov 18 '19 at 12:09
  • @kirk as said in the answer, while ABC is a name for a particular group of methods, "approximate Bayesian inference" is used rather ambiguously. As noticed by Xi'an MCMC seems to differ from VI and ABC, so they do not really fit together under the "approximate" name. Also this would make all the modern Bayesian methods "approximate". – Tim Nov 18 '19 at 13:01
  • I understand that the term "approx. Bayesian inference" is ambiguous hence my question in the first place. I also understand that normally when ppl talk about approx. Bayesian vs Bayesian they usually mean MCMC vs VI. Finally, from your answer I implicity got my answer on my question of wrongfully grouping under the name approx. Bayesian infer. MCMC, VI, Lapl. approx. as the authors did. Thanks! – Kirk Walla Nov 18 '19 at 13:13
  • After asking some statisticians around I've revised my opinion and post it here in hopes for clarity to others as well. There seem to be at least 3 distinct [cases](https://stats.stackexchange.com/questions/4417/what-are-the-factors-that-cause-the-posterior-distributions-to-be-intractable) and ABC falls under 3 while ABI under 2. Which makes my previous remark wrong. According to some statisticians the authors correctly grouped MCMC,VI, LA under ABI, since MCMC is an approximate method regardless its asymptotic consistency, you're trying to approximate a posterior. – Kirk Walla Nov 18 '19 at 15:06
  • @kirk With MCMC you are not trying to approximate posterior, but you are sampling *from* the posterior, that's the difference. – Tim Nov 18 '19 at 15:29
  • Yes I agree, my mistake, you are sampling from the posterior but you don't have the true posterior, do you? So that sampling approach is considered as approximation, at least that's my current understanding. – Kirk Walla Nov 18 '19 at 15:48