1

I already asked this question here, but I am not sure where would be better to ask it? This might sound a dumb question but I am really confused about it. According to Bayes' rule we do have the following: $$p(\theta|X)=\frac{p(\theta)p(X|\theta)}{\int{p(\theta)p(X|\theta)d\theta}}$$I know that probability density function can be greater than one in general but it seems to me because there exist discrete summations of denominator integral which would is greater than the nominator, therefore posterior probability density function cannot be greater than one in any point. Is this correct?!

To explain reasoning a bit in more detail:

Suppose we are interested in $p(\theta=\theta_0|X)$ and we know that: $$\int{p(\theta)p(X|\theta)}d\theta\approx\sum\limits_{n}{p(\theta_i)p(X|\theta_i)}$$ but now only consider the summations which include $p(\theta_0)p(\theta_0|X)$. Then the denominator will obviously be larger than the nominator.

Community
  • 1
Cupitor
  • 1,415
  • 1
  • 11
  • 22
  • 1
    A probability *density* is a probability density, period--and therefore can have values larger than $1$. But if $\theta$ has a *discrete* distribution then $p(\theta|X)$ is not a density, it's a probability. Which situation are you in? – whuber Nov 04 '13 at 22:00
  • see the update. – Cupitor Nov 04 '13 at 22:06
  • I don't understand. Just for asking a question, getting a minus vote?! – Cupitor Nov 04 '13 at 22:10
  • 2
    I'm sorry about that: most people who downvote questions do it with a purpose, and such purposes are only served by also providing a comment explaining the downvote. You and your questions are welcome here. – whuber Nov 04 '13 at 23:02
  • You may be getting a downvote for any number of reasons (e.g. perhaps because the question appears to be asking a oft-repeated duplicate; or perhaps because it appears to be using the term 'density' while also seemingly discussing a discrete distribution). It's better if people explain their downvotes. I will upvote to compensate for the downvote, on the assumption you will work on clarifying the situation of your question some more -- e.g. why do you say 'discrete summation' in respect of the 'denominator integral'?. (If the question remains this ambiguous I might withdraw the upvote later.) – Glen_b Nov 04 '13 at 23:53
  • The short answer (as whuber already said) is that densities can be greater than 1; they're densities, not probabilities. – Glen_b Nov 04 '13 at 23:54
  • @Glen_b, that I do understand(regarding your second comment) and I am not confusing these two. The point is that the integral is nothing more than summation in discrete case, taking the limits to infinity and one is free to choose arbitrary partition of the domain to do the integration. See here: http://en.wikipedia.org/wiki/Riemann_integral#Definition Taking the summation(or more exactly partitions) in the way that I suggest makes the following proportion always smaller than one! So I don't know why a posterior density function might become greater than one at all! – Cupitor Nov 05 '13 at 00:09
  • *integrals* of densities are always less than one. But the densities themselves can exceed one. Consider a uniform on [0.2,0.3] - its density is 10 in between those values ... yet any integral of all or part of the range cannot exceed 1. – Glen_b Nov 05 '13 at 00:14
  • See also the discussion [here](http://stats.stackexchange.com/questions/4220/a-probability-distribution-value-exceeding-1-is-ok). – Glen_b Nov 05 '13 at 00:41
  • 2
    Your second equation is not in general true (even approximately). To see this, consider halving the gaps between the $\theta_i$ and thereby doubling the number of elements in the sum. You will, given some smoothness assumptions, get a sum that is approximately twice that of the original sum (2x the number of elements with about the same average value per element.) Basically, you've left out the "$\text{d}\theta$" part of the integral in the approximating sum. – jbowman Nov 05 '13 at 00:55
  • @jbowman, yupp! thats right! sorry...and thanks a lot. – Cupitor Nov 05 '13 at 00:58

0 Answers0