0

I am studying Bayesian Statistics and one of the things I'm finding challenging is the need to recognize how the likelihoodâ‹…prior maps to a particular distribution, even though it doesn't look exactly like the PDF of the distribution. This is may be done to sample from the posterior without having to solve the difficult or intractable integral used as a normalization factor.

I have spent a good deal of time looking for a resource that teaches this skill, or at least a list of the forms the common distributions might take, but I haven't found anything yet. It seems like Normal, Binomial, Multinomial, Gamma, Beta, and Exponential are all important to be able to recognize.

Does anyone have tips on how to identify the distribution of a posterior or clear resources?

saujosai
  • 1
  • 1

1 Answers1

1

You don't recognize how "likelihood times prior maps to a particular distribution". The distribution obtained from applying Bayes theorem does not need to be any of the named distributions like normal, Poisson, binomial, etc. If you are using a simple model with conjugate prior there would be a simple solution like this. In other cases (most of the cases) there won't be a simple solution and because of that, we use MCMC sampling to approximate the resulting distribution. If we knew the distribution, we wouldn't need to approximate it. The only "mapping" like you describe is the case of the most simple models using conjugate priors. You don't need to "know" the distribution to sample from it, you need only to know it up to a constant, i.e. know the likelihood and a prior

$$ p(\theta | X) \propto p(X|\theta) \, p(\theta) $$

Tim
  • 108,699
  • 20
  • 212
  • 390