4

Let $p$ be some probability distribution with a density $f$.

$p$ is defined over $\Omega$.

Is it true that for any $\omega \in \Omega$, $p(\omega) = 0$?

If not, what are the "minimal conditions" under which $p(\omega) = 0$ for any $\omega$ (the space $\Omega$ is actually a subset of $\mathbb{R}^d$ for some $d$)?

singelton
  • 652
  • 5
  • 13
  • These questions keep coming up. I think this question has been asked before on CV or on both CV and Mathematics. – Michael R. Chernick Oct 11 '12 at 17:34
  • Michael Chernick: Yes, but its a good question and the answer pointed to is a good answer. It has led to numerous mistaken claims of inference anomalies as authors forget they "should not condition on something that was not actually observed such as a continuous outcome, but instead some appropriate interval containing that outcome" page 52 of Cox & Hinkley Theoretical Statistics. – phaneron Oct 11 '12 at 17:50
  • @phaneron Conditioning on events of measure zero is *not always* as issue. An example of this is Bayesian statistics (where, of course, there are pathological examples but that is more the exception than the rule). –  Oct 11 '12 at 17:57
  • 3
    Note that "$p(\omega)$", *sensu stricto,* is not necessarily defined: probabilities can be applied only to *measurable sets* in $\Omega$. Note, too, that the question really applies to the *sigma field* of measurable subsets of $\Omega$, not to $\Omega$ itself. This leaves it open to multiple interpretations; a natural one is, "for what sigma fields $(\Omega, \frak{S})$ do there exist a probability measure $p:\frak{S}\to [0,1]$ having a continuous distribution?" – whuber Oct 11 '12 at 18:03
  • Over $\mathbb{R}$, one definition of "continuous probability distribution" is one such that the probability of any singleton is $0$. This is a strict superset of the absolutely continuous probability distributions, which are defined by probability density functions. An example of a random variable whose distribution is continuous but not absolutely continuous is to take the binary expansion of a uniformly distributed random variable on $[0,1]$, replace every $1$ with a $2$, and consider that string as a ternary expansion. Anyway, both the title and body propositions are true. – Douglas Zare Oct 11 '12 at 21:31
  • @Michael Chernick: I am aware there are ways to enable conditioning on events of measure zero but am wary that issues persist (e.g. Borel Paradox). But in Bayesian statistics (if we agree statistics means dealing with empirical observations) the conditioning (Bayes theorem) is discrete conditioning. If you disagree, perhaps we could make into a question? – phaneron Oct 12 '12 at 12:20

2 Answers2

5

The OP asks for a precise mathematical answer to an imprecise question. For example, the term "probability distribution" has a very precise meaning in terms of the measure transfered by some random object from some probability space to its image, but here you have only one probability space, and no random object. I will try to give an answer in his original setting.

Suppose that you have a measurable space $(\Omega,\mathscr{F}$), where $\Omega=\mathbb{R}$ and $\mathscr{F}$ is the class of Borel subsets of $\mathbb{R}$. The extension to higher dimensions adds nothing conceptually.

You have a probability measure $P$ over $(\Omega,\mathscr{F}$), and you say that $P$ has a density $f$. What does that mean exactly? It means that you have a measure (not necessarily a probability measure) $\lambda$ over $(\Omega,\mathscr{F}$) and $$ P(B) = \int_B f(\omega)\,d\lambda(\omega) \, , $$ for every $B\in\mathscr{F}$, where the nonnegative $f$ is such that $\int_\mathbb{R}f\,d\lambda=1$.

The measure $\lambda$ dominates $P$, in the sense that for every $B\in\mathscr{F}$, if $\lambda(B)=0$ then $P(B)=0$.

Note that for each $\omega\in\Omega$, the singleton $\{\omega\}\in\mathscr{F}$, so it is legal to talk about $P(\{\omega\})$, and that is the substance of the point raised by the comment made by whuber.

Now, to answer your question "Is it true that $P(\{\omega\})=0$ for every $\omega\in\Omega$?", consider two cases.

  1. Let $\lambda$ be Lebesgue measure. In this case, the answer is clearly "Yes", because $\lambda(\{\omega\})=0$ for every $\omega\in\Omega$, and $\lambda$ dominates $P$.

  2. Let $\lambda$ be such that $\lambda(\mathbb{R})=2$, and $\lambda(\{0\})=\lambda(\{1\})=1$. Let a function $f$ be defined by $f(0)=1/4$, $f(1)=3/4$, and define $f(\omega)$ arbitrarily for $\omega\notin \{0,1\}$. Now, define a probability measure $P$ over $(\Omega,\mathscr{F})$ by $$ P(B) = \int_B f(\omega)\,d\lambda(\omega) \, . $$ This $P$ is a genuine probability measure, it has density $f$ with respect to $\lambda$, but $$ P(\{0\}) = \int_{\{0\}} f(\omega)\,d\lambda(\omega) = f(0) \cdot \lambda(\{0\}) = \frac{1}{4} \, , $$ and $$ P(\{1\}) = \int_{\{1\}} f(\omega)\,d\lambda(\omega) = f(1) \cdot \lambda(\{1\}) = \frac{3}{4} \, , $$ Hence, in this case the answer is "No".

Short answer: it depends on the dominating measure.

Zen
  • 21,786
  • 3
  • 72
  • 114
1

I'd like to share the answer to this question with you in very simple words. Because I also wondered in my early days on why R gives a value when I input a single value in any density function of a continuous variable, say for example in dnorm. As I know for a particular value the probability should be zero.

Here an important thing to be noticed is that if you manually put the value of the random variable, say, $X$, in the form of the probability density function (p.d.f.) of a continuous random variable, then you do not get the probability. What you get is called simply the 'ordinate'. If you consider a X-Y plane, then this value is the $Y$ coordinate value against your value for $X$ coordinate.

Following is a practical example of why a particular point gives probability zero in a continuous probability distribution:

While for a discrete distribution an event with probability zero is impossible (e.g. rolling 3½ on a standard die is impossible, and has probability zero), this is not so in the case of a continuous random variable. For example, if one measures the width of an oak leaf, the result of 3½ cm is possible, however it has probability zero because there are uncountably many other potential values even between 3 cm and 4 cm. Each of these individual outcomes has probability zero, yet the probability that the outcome will fall into the interval (3 cm, 4 cm) is nonzero. This apparent paradox is resolved by the fact that the probability that $X$ attains some value within an infinite set, such as an interval, cannot be found by naively adding the probabilities for individual values. Formally, each value has an infinitesimally small probability, which statistically is equivalent to zero.

Blain Waan
  • 3,345
  • 1
  • 30
  • 35
  • @Procrastinator: Would you kindly explain a little about the value 'dgamma', 'dnorm' etc. give? I hope this will help me to learn it. Besides, I have deleted the part that was confusing. Your explanation will certainly enrich the answer. – Blain Waan Oct 11 '12 at 18:20
  • 2
    `dnorm` gives you the height of the density function at that point, not the probability of drawing that point which would be the area under the density function. – wcampbell Oct 11 '12 at 18:27
  • @wcampbell: So, 'dnorm' will give me what I mentioned in my 2nd para? Not the area under the curve, have I got it now? – Blain Waan Oct 11 '12 at 18:31
  • @Blain Some relevant discussion about this was prompted by a thread on why [probability densities can exceed 1](http://stats.stackexchange.com/questions/4220/probability-distribution-value-exceeding-1-is-ok). I included an account of the infinitesimal point of view in a reply to a different question at http://stats.stackexchange.com/a/14490. – whuber Oct 11 '12 at 18:44
  • 2
    "the probability that X attains some value within an infinite set, such as an interval, cannot be found by naively adding the probabilities for individual values." While this statement is true for the special case of intervals (as well as for measurable sets containing an uncountably infinite number of points), it is not true for _countably_ infinite sets where adding the probabilities _does_ in fact work. Countable sums of numbers can be defined in terms of limits (when they exist) and the "naive" method of adding probabilities breaks down because uncountable sums are not defined. – Dilip Sarwate Oct 11 '12 at 22:37