Questions tagged [probability]

A probability provides a quantitative description of the likely occurrence of a particular event.

Overview

Probability is conventionally expressed on a scale from $0$ to $1$: a rare or unlikely event has a probability close to $0$ while a common or expected event has a probability close to $1$.

The notion of probability has been shaped by Andrey Kolmogorov and his Axioms of Probability. For a sample space $\Omega$ and a sigma algebra $S$, a probability function $P$ satisfies

  1. $P(A) \geq 0$ for all $A \in S$.
  2. $P(\Omega) = 1$.
  3. When $A_1,A_2,... \in S$ are pairwise disjoint, $P\left( \cup_{i=1}^{\infty} A_i \right) = \sum_{i=1}^{\infty}P(A_i).$

These axioms form a set of useful rules for calculating probabilities.

The probability of an event has been interpreted variously as its long-run relative frequency and as a personal degree of belief (subjective probability).

References

The following threads on math.se contain references to resources about probability:

11025 questions
607
votes
12 answers

What is the difference between "likelihood" and "probability"?

The wikipedia page claims that likelihood and probability are distinct concepts. In non-technical parlance, "likelihood" is usually a synonym for "probability," but in statistical usage there is a clear distinction in perspective: the number that…
Douglas S. Stones
  • 6,931
  • 4
  • 16
  • 18
280
votes
16 answers

Why does a 95% Confidence Interval (CI) not imply a 95% chance of containing the mean?

It seems that through various related questions here, there is consensus that the "95%" part of what we call a "95% confidence interval" refers to the fact that if we were to exactly replicate our sampling and CI-computation procedures many times,…
Mike Lawrence
  • 12,691
  • 8
  • 40
  • 65
182
votes
4 answers

Why do we need sigma-algebras to define probability spaces?

We have a random experiment with different outcomes forming the sample space $\Omega,$ on which we look with interest at certain patterns, called events $\mathscr{F}.$ Sigma-algebras (or sigma-fields) are made up of events to which a probability…
Antoni Parellada
  • 23,430
  • 15
  • 100
  • 197
174
votes
6 answers

Can a probability distribution value exceeding 1 be OK?

On the Wikipedia page about naive Bayes classifiers, there is this line: $p(\mathrm{height}|\mathrm{male}) = 1.5789$ (A probability distribution over 1 is OK. It is the area under the bell curve that is equal to 1.) How can a value $>1$ be OK? I…
147
votes
15 answers

Amazon interview question—probability of 2nd interview

I got this question during an interview with Amazon: 50% of all people who receive a first interview receive a second interview 95% of your friends that got a second interview felt they had a good first interview 75% of your friends that DID NOT…
Rick
  • 1,431
  • 2
  • 11
  • 9
140
votes
21 answers

What's the difference between probability and statistics?

What's the difference between probability and statistics, and why are they studied together?
hslc
  • 1,537
  • 3
  • 10
  • 3
125
votes
9 answers

Numerical example to understand Expectation-Maximization

I am trying to get a good grasp on the EM algorithm, to be able to implement and use it. I spent a full day reading the theory and a paper where EM is used to track an aircraft using the position information coming from a radar. Honestly, I don't…
119
votes
20 answers

At each step of a limiting infinite process, put 10 balls in an urn and remove one at random. How many balls are left?

The question (slightly modified) goes as follows and if you have never encountered it before you can check it in example 6a, chapter 2, of Sheldon Ross' A First Course in Probability: Suppose that we possess an infinitely large urn and an infinite …
Carlos Cinelli
  • 10,500
  • 5
  • 42
  • 77
115
votes
16 answers

If 900 out of 1000 people say a car is blue, what is the probability that it is blue?

This initially arose in connection some work we are doing to a model to classify natural text, but I've simplified it... Perhaps too much. You have a blue car (by some objective scientific measure - it is blue). You show it to 1000 people. 900 say…
Pat Molloy
  • 1,115
  • 2
  • 7
  • 6
114
votes
4 answers

What does a "closed-form solution" mean?

I have come across the term "closed-form solution" quite often. What does a closed-form solution mean? How does one determine if a close-form solution exists for a given problem? Searching online, I found some information, but nothing in the context…
95
votes
6 answers

Convergence in probability vs. almost sure convergence

I've never really grokked the difference between these two measures of convergence. (Or, in fact, any of the different types of convergence, but I mention these two in particular because of the Weak and Strong Laws of Large Numbers.) Sure, I can…
raegtin
  • 9,090
  • 12
  • 48
  • 53
91
votes
6 answers

If I have a 58% chance of winning a point, what's the chance of me winning a ping pong game to 21, win by 2?

I have a bet with a co-worker that out of 50 ping pong games (first to win 21 points, win by 2), I will win all 50. So far we've played 15 games and on average I win 58% of the points, plus I've won all the games so far. So we're wondering if I have…
richard
  • 881
  • 1
  • 7
  • 5
91
votes
7 answers

Why to optimize max log probability instead of probability

In most machine learning tasks where you can formulate some probability $p$ which should be maximised, we would actually optimize the log probability $\log p$ instead of the probability for some parameters $\theta$. E.g. in maximum likelihood…
Albert
  • 1,145
  • 1
  • 9
  • 12
81
votes
9 answers

Probability of a single real-life future event: What does it mean when they say that "Hillary has a 75% chance of winning"?

As the election is a one time event, it is not an experiment that can be repeated. So exactly what does the statement "Hillary has a 75% chance of winning" technically mean? I am seeking a statistically correct definition not an intuitive or…
pitosalas
  • 933
  • 1
  • 7
  • 6
79
votes
8 answers

Is there a name for the phenomenon of false positives counterintuitively outstripping true positives

It seems very counter intuitive to many people that a given diagnostic test with very high accuracy (say 99%) can generate massively more false positives than true positives in some situations, namely where the population of true positives is very…
Roger Heathcote
  • 893
  • 1
  • 4
  • 6
1
2 3
99 100