Posterior becomes infinity for bayes-theorem interpretation

Question

posterior $=$ $\frac{likelihood * prior}{evidence}$

If evidence = 0, then posterior is infinity or doesn't exist.

What does that mean in terms of posterior, likelihood, prior, etc? What happens to the likelihood and prior when the evidence is zero? Do they also become zero or nonexistent?

evidence = 0 -> you saw an impossible result from your experiment. Think about what *that* means about your likelihood, prior, etc.! — jbowman, Jan 24 '20 at 19:06
https://stats.stackexchange.com/questions/108797/in-naive-bayes-why-bother-with-laplace-smoothing-when-we-have-unknown-words-in — independentvariable, Jan 24 '20 at 19:07

score 1 · Accepted Answer · answered Jan 24 '20 at 21:07

First of all, division by zero is not infinity, but undefined. Being more specific about your question, Bayes theorem is

$$ p(\theta|X) = \frac{p(X|\theta)\;p(\theta)}{\int p(X|\theta)\;p(\theta)\; d\theta} $$

so to see exact zero in the denominator, you would also need to see zeroes in the numerator, either in the likelihood $p(X|\theta)$, or prior $p(\theta)$. As stated in comments, this would mean that you observed data that is impossible under your model. Such situation should never happen, it suggests that either your model is wrong, or as discussed in one of the threads in the comments, it is a problem with insufficient data that needs to be handled.

Posterior becomes infinity for bayes-theorem interpretation

1 Answers1