10

I know this is duplicated but I think the question is a bit different and needs different answer.

How can CDF be continuous and have derivative at each point that is not equal to zero but the probability at each point is zero?

Why not say for example if you want to choose a real number between 0 and 1 that the probablity is 1/N? Where this is just infinitesimal. (Just an example to describe the situation.)

It can neither be finite nor zero

The difference is: Why not assign it an infinitesimal instead of zero and to say that an event with P = 0 can happen?

Glen_b
  • 257,508
  • 32
  • 553
  • 939
user3733086
  • 101
  • 1
  • 1
  • 5
  • What does N stand for? I do not follow your logic and I don't see how you have really made this a different question. – Michael R. Chernick Apr 12 '17 at 17:55
  • N refers to the number of real number between 0 and 1, Which seem ridiculous because it is infinite but Why do we assign it to zero? And if it is zero you would expect cdf to be a horizontal line with zero derivative? Why not assign it as a infinitesimal just as we do we derivative – user3733086 Apr 12 '17 at 17:59
  • 2
    There is a continuum of real numbers between 0 and 1. It is not only infinite it is uncountably infinite. – Michael R. Chernick Apr 12 '17 at 18:02
  • So how can I solve this problem? I have been reading and thinking about it for days. – user3733086 Apr 12 '17 at 18:09
  • 3
    This is really a question of understanding the mathematics (calculus) that statistics & probability are built on, rather than primarily a probability question. It might help you to read this [math.SE] SE question: [Is dy/dx not a ratio?](http://math.stackexchange.com/q/21199) – gung - Reinstate Monica Apr 12 '17 at 18:11
  • Yes I have read that link before, I read a lot about the history of calculus and the arguments against infinitesimal and limits. I do like the idea of infinitesimal, I will continue using it until I find where it becomes a error ( If it does) and you need more rigorous math about it ( non-standard). The question here is whether I can apply this fact to probabilities because it doesn't make sense at all to say it is zero. Yes, there are infinite choices But if you make eg a program that chooses a number then the P isnt zero. Or you can say that it is zero but still can happen (Contradiction) – user3733086 Apr 12 '17 at 18:26
  • Lets say natural numbers instead – user3733086 Apr 12 '17 at 18:26
  • 2
    A computer program cannot draw any real number in $[0,1]$. It can't even draw any rational number, only the set of possible rational numbers that can be represented in the finite number of bits assigned to the variable. Can you name me any real world example in which we draw a number from a continuous distribution, whose value is known exactly? – Bridgeburners Apr 12 '17 at 19:04
  • "Lets say natural numbers instead" ??? it is not continuous but it is infinite. – user3733086 Apr 12 '17 at 19:08
  • Can you think of a scenario where a number is drawn uniformly from the set of all natural numbers? Does that make sense? Just think about that for a moment. If $X$ is supposedly distributed uniformly over all natural numbers, what do you expect a finite sample of draws of the variable $X$ to look like? – Bridgeburners Apr 12 '17 at 19:23
  • If you deal with infinities then I guess you have to use infinities to support your claim, If I do it infinitely many times I should get a uniform distribution. Any finite stopping will cause it to have no distribution or at least doesnt make sense. Paradoxes! – user3733086 Apr 12 '17 at 19:30
  • 1
    Please read the first third of Edward Nelson's pretty little monograph, [Radically Elementary Probability Theory](https://web.math.princeton.edu/~nelson/books/rept.pdf). – whuber Apr 12 '17 at 22:01
  • this might be interesting: http://math.stackexchange.com/questions/257655/hyperreal-measure – jld Apr 12 '17 at 22:24

3 Answers3

14

A guess at your point of confusion:

Zero probability does not mean an event cannot occur! It means the probability measure gives the event (a set of outcomes) a measure zero.

As @Aksakai's answer points out, the union of an infinite number of zero width points can form a positive width line segment and similarly, the union of an infinite number of zero probability events can form a positive probability event.

More explanation:

  • Our intuition from discrete probability is that if an outcome has zero probability, then the outcome is impossible. If the probability of drawing the ace of spades from a deck is equal to zero, it means that ace of spades is not in the deck!

  • With continuous random variables (or more generally, an infinite number of possible outcomes) that intuition is flawed.

    • Probability measure zero events can happen. Measure one events need not happen. If an event has probability measure 1, you say that it occurs almost surely. Notice the critical word almost! It doesn't happen surely.
    • If you want to say an event is impossible, you may say it is "outside the support." What's inside and outside the support is a big distinction.
    • Loosely, an infinite sum of measure zero events can add up to something positive. You need an infinite sum though. Each point on a line segment has zero width, but collectively, they have positive width.
Matthew Gunn
  • 20,541
  • 1
  • 47
  • 85
  • "the probability of such an outcome occurring is smaller than any fixed positive probability, and therefore must be 0", Also the example of darts. Isn't it better to say it is infinitesimal better than zero? It means that it can happen and the sum of it can be equal to 1. That is my question If you consider each point as zero, then it doesn't make sense at all to define zero as being that it can happen. About the calculus part, If it works and intuitive, Why not use it? I think limits didn't solve the problem of infinitesimal. Everyone uses infinitesimal to make argument but they use limits – user3733086 Apr 12 '17 at 19:25
  • 2
    "Zero probability does not mean impossible" - I always had trouble with this. I'd say if it happened then probability is not zero. – Aksakal Apr 12 '17 at 20:15
  • 1
    @Aksakal: Suppose that you flip a fair coin repeatedly, stopping when it lands heads. I make two claims: (a) it is possible that you could keep on flipping the coin forever, with the coin landing tails infinitely many times; (b) the probability that the coin lands tails infinitely many times is zero. With which one do you disagree? – wchargin Apr 12 '17 at 20:21
  • 1
    I would disagree with b, I think people only say zero because it is the most suitable number in real numbers, it should be a number that satisfies e> 0 where e should be really close to zero. The only number that may satisfies that is zero. That is why I argue that infinitesimal are better – user3733086 Apr 12 '17 at 20:38
  • 2
    @wchargin You can't observe infinite number of heads – Aksakal Apr 12 '17 at 20:40
  • @user3733086 in the case of infinite sequences of coin tosses we're taking the state space $\Omega$ to be the set of all $\aleph_0$-length binary sequences and we're discussing the probability of the event $\{(1, 1, \dots)\}$. This probability is exactly 0 in the standard formulation with real-valued (rather than hyperreal-valued) measures – jld Apr 12 '17 at 22:23
  • Can you clarify what you mean by "Zero probability does not mean an event cannot occur!"? I.e., what does "Zero probability" mean for a continuous random variable? For a concrete example, suppose $X$ is a normally distributed random variable. Then does $P[X=5]=0$ just mean that, if I have many draws (independent realizations) of $X$ the number $5$ will appear relatively infrequently (i.e. $\frac{\#5}{N} \approx 0$ for large $N$, where $N$ is the number of draws and $#5$ is the number of 5's we see)? – user106860 May 15 '19 at 02:38
  • 1
    @user106860 Saying a probability measure assigns an event a measure of zero is equivalent to saying the event [almost surely](https://en.wikipedia.org/wiki/Almost_surely) will not occur. Zero probability (along with law of large numbers and some more argument) indeed implies that your $\lim_{N \rightarrow \infty} \frac{\#5}{N} = 0$, but that doesn't explain what's going on. Aksakal's answer gets more at the heart of the matter. The concepts in play here are really concepts from set theory and real analysis. – Matthew Gunn May 16 '19 at 12:49
  • @user106860 For a continuous random variable such as a standard normal distributed $X$, the probability of $X=c$ is 0 for every $c$ (i.e. $P(X=c)=0$ for all $c \in \mathbb{R}$). If 0 probability implied an event cannot occur, then every outcome of $X$ could not occur (which is obviously non-sensical and false). – Matthew Gunn May 16 '19 at 17:01
12

It's really not a statistics question. It's a real analysis question. For instance, it's almost the same as asking "what's the width of a point in line?" (the answer is zero, by the way)

This is an interesting situation though. In mathematics the line is defined as a set of points. There are certain geometric constraints on the points, so that they form a line and not a circle, for instance. However, that's not what's important.

What's important is this. If the width of each point is zero, and the line is a set of points, how come the sum of widths of all its points is NOT zero? You add two zeros and it gives you a zero. If I keep adding this way shouldn't the length of a line be zero? Apparently, not!

This is the same question you're asking. How is it that each point's probability is zero, yet the total probability is one? The reason why is this question the same is because probabilities are intimately linked to the concept of the length of a line between two points. The central concept of the modern probability theory is the concept of a measure. Unsurprisingly, it has its roots in the simplest of all measures: the length in geometry.

If you want a shortcut in understanding this bind boggling stuff then look up the concept of countable and uncountable sets. Note the difference between infinite countable sets and uncountable sets. Both have infinite number of points in them, yet the latter has more points in it (totally crazy!). So discrete and continuous random variables (and their distributions) are related to these two kinds of sets.

UPDATE
Example: In English there are countable and uncountable nouns such as apple vs. milk. I could ask you how much does an apple weigh? And you could say that it's half a pound in this batch. However, if I asked how much does milk weigh, it wouldn't make a sense without specifying an amount such as a pint or a quart.

In this regard the discrete random variables and their probabilities are like apples and their weights. You could say that the probability of Poisson variable 1 is 10%, for instance.

The continuous random variables are like milk. It's pointless asking what's the probability of a given value, you need to specify the bucket. Say, for a standard normal (Gaussian) variables you could ask what's the probability that their values are between 0 and 1, and the answer would be something like 34%. However, the probability of 1 is pretty much meaningless in practical sense. You can calculate the density at $x=1$ but what are you going to do with it? It's not the probability. In the same way if you're interested in the weight of milk the density of milk is not an answer, you need to specify the container size then we can tell you the weight using its density. That's why probability density function is actually called density, it originates from densities of bodies.

gung - Reinstate Monica
  • 132,789
  • 81
  • 357
  • 650
Aksakal
  • 55,939
  • 5
  • 90
  • 176
  • Really really interesting argument..This is the dark side of mathematics. "almost the same" but in a sense points should have zero width and probabilities shouldn't. I don't really know how you can define a length as the sum of width of points, I don't think that it is a good definition...? – user3733086 Apr 12 '17 at 20:21
  • 2
    @user3733086, the width of a point on a curve and the probability density have the same math behind. – Aksakal Apr 12 '17 at 20:39
  • And there is no solution to this paradox?? – user3733086 Apr 12 '17 at 20:53
  • 1
    @user3733086, it's not a paradox. It's one of the first things they teach in real analysis, then continue in measure theory. I'm not sure how to explain it intuitively. I was just trying to point out that it's not a stat question really hoping that you'd look at the math behind it – Aksakal Apr 12 '17 at 21:01
  • @user3733086 The solution is to stop thinking measure zero is synonymous with impossible and start thinking that measure zero simply means something has measure zero. For example, the [Lebesgue measure](https://en.wikipedia.org/wiki/Lebesgue_measure) of the open interval $(2, 3)$ is 1. The Lebesgue measure of the closed interval $[2,3]$ is also 1 (even though the latter is a superset of the former). The Lebesgue measure of the point $2$ and the point $3$ is zero. – Matthew Gunn Apr 12 '17 at 21:02
  • @MatthewGunn, Lebesque measure is basically the length or width but in a very abstract sense. So if someone has issues with width then Lebesque will be too much – Aksakal Apr 12 '17 at 21:09
  • @user3733086 let $\{q_1, q_2, \dots\}$ be countable and let $\lambda$ be the Lebesgue measure. We know that $\lambda(\bigcup_i \{q_i\}) = \sum_i \lambda(\{q_i\}) = 0$ by the $\sigma$-additivity of $\lambda$, so we can't get something "solid" with countable unions of points. We need an uncountable union and $\lambda$ is not uncountably-additive, so that's why for instance $1 = \lambda([0,1]) \neq \bigcup_{r \in [0,1]} \lambda(\{r\}) = 0$ – jld Apr 12 '17 at 22:38
  • +1 for the milk example (though not only). – Richard Hardy Jan 07 '21 at 20:25
  • Apparently this is so mind boggling that you invented a new word for it: “bind boggling”. So there are things that are mind boggling, and then there are things that are so beyond mind boggling, they’re bind boggling! – lostsoul29 May 20 '21 at 00:27
0

I think it is helpful to imagine the area under the point. The probability for a continuous distribution is the integral of the PDF from (a,b). If you pick a single point (a,a) is there any area? Imagine a simple PDF like the uniform distribution do the math.

PS No, but if you ask enough mathematicians 1/20 will say yes. However, I'll accept the null with an $\alpha$ of 5%.

Matt L.
  • 739
  • 3
  • 10