6

When talking about events, there is the following formula called Bayes' rule, where $A$ and $B$ are random events: $$P(A|B)=\frac{P(B|A)P(A)}{P(B)}$$

Now let's say that for now only $A$ happened. I suppose that $P(B|A)$ is called posterior probability (beside conditional probability) since there is an evidence, $A$.

But is $P(A|B)$ also called posterior probability?

Edit: An (artificial) example:

$A$ = "Last year I had no accidents."

$B$ = "This year I will have no accidents."

Edit 2:

Simply put: $P(\text{something}|\text{evidence})$ = posterior probability

But $P(\text{evidence}|\text{something})$ = posterior?

Doesn't "posterior" have something to do with the chronology of events (their order of happening in time)?

Mussé Redi
  • 103
  • 4
user_anon
  • 827
  • 10
  • 20
  • Posterior *is* a conditional probability. Could you try making your question more clear? What are $A$ and $B$? Maybe you could illustrate it with an example? – Tim May 22 '18 at 09:32
  • $A$ and $B$ are random events. – user_anon May 22 '18 at 09:37
  • What my question was about: P(something|evidence) = posterior pb. But P(evidence|something) = posterior? – user_anon May 22 '18 at 09:39

1 Answers1

14

TLDR; Posterior probability is just the conditional probability that is outputted by the Bayes theorem. There is nothing special about it, it does not differ anyhow from any other conditional probability, it just has it's own name.

The Bayes theorem is about obtaining one conditional probability $P(A|B)$, given another one $P(B|A)$ and the prior $P(A)$,

$$ \underbrace{P(A|B)}_\text{posterior}=\frac{P(B|A)\,\overbrace{P(A)}^\text{prior}}{P(B)} $$

So in the equation we have two random variables $A$ and $B$ and their conditional and marginal probabilities, that's all. Prior $P(A)$ is the probability of $A$ "before" learning about $B$, while posterior $P(A|B)$ is the probability of $A$ "after" learning about $B$, where the "before" and "after" refer to your procedure of calculating the probabilities, not any chronological order. The naming convention is that the left hand side is the posterior, while the prior appears in the right hand side part. Using Bayes theorem you can easily switch the sides back and forth (that's the point of the theorem). The usual use case is when you know only $P(B|A)$ and $P(A)$, but you don't know $P(A|B)$ and want to learn about it. Here you can find nice example of such usage of Bayes theorem.

The specific case is Bayesian inference, where we use Bayes theorem to learn about the distribution of the parameter of interest $\theta$ given the data $X$, i.e. obtain the posterior distribution $f(\theta|X)$. This is achieved by looking at the likelihood function (that looks at the "evidence" you gathered) and the prior (the distribution of $\theta$ that is assumed before looking at the data).

$$ \underbrace{f(\theta|X)}_\text{posterior}=\frac{\overbrace{f(X|\theta)}^\text{likelihood}\,\overbrace{f(\theta)}^\text{prior}}{\underbrace{f(X)}_\text{normalizing constant}} $$

The most basic example of such usage of Bayes theorem is the beta-binomial model, but Bayesian inference is not only about probabilities of coin tosses, for example you can use it to do regression analysis and many other, even much more complicated, models.

To comment on your last question:

Doesn't "posterior" have something to do with the chronology of events (their order of happening in time)?

Not at all. You can ask a valid probabilistic question is reverse time order, e.g. "What is the probability that it was raining in the morning given that the ground is wet?".

Tim
  • 108,699
  • 20
  • 212
  • 390
  • So, when not doing Bayesian inference, the posterior and prior "exist" only when talking about Bayes theorem? Without considering the Bayes theorem with $P(A|B)=\ldots$, I cannot state that $P(A|B)$ is a posterior probability, right? – user_anon May 22 '18 at 11:02
  • @user_anon Bayes theorem is about obtaining posterior probability $P(A|B)$ given prior $P(A)$ and the likelihood $P(B|A)$. Posterior probability and the likelihood are both conditional probabilities. What I'm saying is that most of the descriptions you'll see are about specific usage of Bayes theorem in Bayesian inference, where the priors are "assumed" probabilities, while likelihood is probability of observing your data, so they have pretty specific meaning. – Tim May 22 '18 at 11:10
  • I understand that most of the time when talking about posterior probability it is about Bayesian inference. But let's say we consider the other cases. Now, $P(A|B)$ is a conditional probability. $P(A|B)$ is not a posterior probability because I didn't say anything about the way of computing it. If I know $P(B|A),P(A),P(B)$ and plug those into the Bayes theorem, then $P(A|B)$ is also a posterior probability. Right? – user_anon May 22 '18 at 11:16
  • 2
    @user_anon posterior probability is just the conditional probability that is outputted by the Bayes theorem. There is nothing special about it, it does not differ anyhow from any other conditional probability, it just has it's own name. – Tim May 22 '18 at 11:19
  • I found a note in a course: "$P(A|B)=\frac{P(A,B)}{P(B)}$, Note: $P(A|B)$ is called a posteriory probability of $A$, given $B$." Is it correct? – user_anon May 22 '18 at 11:25
  • 1
    @user_anon It's the same since $P(A, B) = P(B|A)P(A)$ see https://en.wikipedia.org/wiki/Conditional_probability – Tim May 22 '18 at 11:26
  • So, anywhere I see the Bayes theorem (or conditional probability definition formula) I can spot the posterior probability as the output of the formula. And only there! Right? – user_anon May 22 '18 at 11:46
  • @user_anon Exactly. However, you can use one posterior as a prior for another calculation by employing the chain rule of probability: https://en.wikipedia.org/wiki/Chain_rule_(probability) , so don't assume that something stays "posterior" forever. – Tim May 22 '18 at 11:51
  • Ok. I understand your point. But a prior doesn't have to be "unconditional"? A posterior is always conditional... – user_anon May 22 '18 at 11:55
  • 1
    @user_anon this happens when you chain together many Bayes theorem equations, but it's a different story and unrelated to your question. – Tim May 22 '18 at 12:04
  • http://web.mit.edu/neboat/Public/6.042/conditionalprobability.pdf :( page 6, first sentence – user_anon Aug 13 '18 at 15:18
  • @user_anon I don't understand your comment. This is what I was saying. – Tim Aug 13 '18 at 15:26
  • In the pdf it says: "A conditional probability Pr (B | A) is called an a posteriori if event B precedes event A in time.". You said that a posteriori has nothing to do with time. (I still believe you :) ) – user_anon Aug 13 '18 at 15:31
  • 1
    @user_anon Because it doesn't. You can condition on time preceding event but do not have to. You can calculate probability that it was raining given that the ground is wet now. – Tim Aug 13 '18 at 15:54