When the large number law fails, the frequency of an event does not converge to the probability, so what is the meaning of probability in these cases? PS: Please adds references to the answers (if it possible).
Asked
Active
Viewed 540 times
0
-
4An answer to [your previous question](http://stats.stackexchange.com/questions/29882/when-does-the-law-of-large-numbers-fail) demonstrates that it cannot possibly fail in this application (for which it is implicitly assumed the repeated observations of the event are *independent*), so this question is devoid of content. – whuber Jun 06 '12 at 18:28
-
The failure of a law of large numbers does not imply there is an event for which the observed frequency does not converge to the probability. For example, the failure of the sample mean of a Cauchy variate to converge to anything says something about a particular integral, but not about long run frequency $\to$ probability. Also what @whuber said (+1). – jbowman Jun 06 '12 at 19:47
-
I am not sure what this question is actually asking. If it is a philosophy of mathematics/statistics question I do not think Cross Validated is the right place to ask it. If the question of how to interpret I.I.D random variables when some portion of the hypotheses of a LLN fails you might want to give an example of what you mean (I am having trouble thinking of any example off the top of my head). – Fraijo Jun 06 '12 at 19:16
-
1We seem to field a fair number of philosophical questions, so I wouldn't be too concerned it's off-topic. But, under standard assumptions, it is a bit vacuous. – cardinal Jun 06 '12 at 19:32
3 Answers
3
As pointed out in the comments and as discussed here, it is extremely difficult to think of practical examples where the LLN fails to hold for the probability of an event. I think this illustrates the strength of Kolmogorov's result.
It is also important to point out that it pacified the debate between Bayesian and frequentists. It means that you do not need to worry about the question "what is a probability?", but only about "can I measure it?".
Still, there is one obvious case when it is impossible to get IID sampling of an event: when it happens only once. In that case the LLN does not apply and we are back to good old debate on what is a probability. But I think this is also outside the scope of this forum so I'll just cite some references from the top of my head.
Henri Poincaré in Science et Méthodes (1914) emphasized the deteministic nature of randomness. Chance is an effect of the imprecision of our measure instruments.
Paul Lévy in Calcul des probabilités (1925) advocates that probabilities of single events are meaningful only if they are close to 0 and 1.
Bruno de Finetti proposed to define probability as a coherent weighing scheme of prior beliefs.
More recently, Peter Walley in Statistical reasoning with imprecise probabilities tried to extend de Finetti's definition to make probability a representation of human knowledge.
This list is obviously very biased and incomplete to say the least.
gui11aume
- 13,383
- 2
- 44
- 89
0
I think you are confusing events with random variables. The strong law of large numbers says that if $S_n= (X_1+X_2+X_3+...+X_n)$ then $S_n/n$ converges to a constant almost surely. This is convergence of a sequence of random variables to a random variable that is degenerate (i.e. equal to a constant). If the law of large numbers fails the convergence could be to a non-degenerate distribution or it may just not converge at all. But this does not connect to relative frequencies of measurable events in a probability space. Suppose $A$ is a measureable event in a probability space. Define the independent binary variables $X_i$ for $i=1,2,3,..,n$ such that $X_i=1$ if $A$ occurs and $X_i=0$ otherwise. Then $S_n$ is Binomial in this case and $S_n/n$ does converge to $p=P(A)=E(X_i)$. So there is no contradiction about measurable events having their probaility expressible as a limiting relative frequency.
Michael R. Chernick
- 39,640
- 28
- 74
- 143
-
2I think the sentence *But this does not connect to relative frequencies of measurable events in a probability space* is (potentially) confusing, especially since you pretty much demonstrate the opposite immediately following. :) – cardinal Jun 06 '12 at 17:11
-
I said that in the context of failure of the law of large numbers not implying failure of a frequency of occurrence of events to converge to the probability of the event. That is where the OP seems to be confused. At least that is what i read out of his question. The sequence of random variables that fail to converge does not connect to the occurrence or nonoccurrence of a measurable event. But throught the bernoulli variable that I defined you can connect it to a sequence of random variables but for that sequence the strong law applies. – Michael R. Chernick Jun 06 '12 at 17:40
0
The Law of Large numbers as it is commonly defined doesn't deal with events converging to a probability but with the results of events converging to an average value.
Let's take an example of a die with six sides.
According to the law of large numbers the average is supposed to converge towards $\frac{1+2+3+4+5+6}{6}=3.5$. A probability is a value between 0 and 1. The 3.5 in this example is no probability. The probability that a single dice hits 3.5 is even zero.
-
3That "relative frequencies" converge almost surely to the true probability of the event is a particular *subcase* covered by the LLN, though. – cardinal Jun 06 '12 at 23:51