4

I am bit puzzled on how we can interpret the posterior. Assume a coin which is 0.1 probable to be unfair. So our prior probability on the coin being unfair is 0.1, and being fair is 0.9. Also by unfair I mean, the probability of head is 2/3 instead of 1/2. Now, imagine a scenario where I toss this coin 10 times and I get 10 heads. So if I want to get the posterior, I do the following:

$$ P(\texttt{unfair}|\texttt{evidence}) = \frac{P(\texttt{evidence}|\texttt{unfair})*P(\texttt{unfair})} {P(\texttt{evidence})} = \frac{P(\texttt{evidence}|\texttt{unfair})*P(\texttt{unfair})} {P(\texttt{evidence}|\texttt{unfair})*P(\texttt{unfair})+P(\texttt{evidence}|\texttt{fair})*P(\texttt{fair})} =\\ \frac{(2/3)^{10}*(0.1)}{(2/3)^{10}*(0.1)+(1/2)^{10}*(0.9)} = \frac{0.0017}{0.0017+0.0009} = \frac{0.0017}{0.0026} = 0.653$$

Now how can I interpret this posterior? It's technically saying that the probability of the coin being unfair given the evidence is higher than the prior belief on the coin being unfair (0.653 > 0.1), but is still below the prior belief on the coin being fair (0.653 < 0.9). So, the only conclusion is, the coin is still more likely to be fair. So, in order to change our belief about the coin we have "being unfair", we need more evidence (i.e. 100 heads in a row). Is it correct?

Glen_b
  • 257,508
  • 32
  • 553
  • 939
user3639557
  • 1,042
  • 10
  • 21
  • 2
    Note that enclosing LATEX inside of dollar signs $ renders it as math. For example: `\alpha` is $\alpha$. – Sycorax Apr 27 '15 at 02:51

1 Answers1

6

Using the same approach, you can compute the posterior probability that the coin is fair. (Do the exercise!) What do you make of the result?

user3697176
  • 852
  • 4
  • 10
  • 2
    $$\frac{0.0009}{0.0026} = 0.34$$ which is lower than 0.653. Based on the updated posterior for fairness, we can conclude the coin is more likely to be unfair? – user3639557 Apr 27 '15 at 04:28
  • It should actually be 0.347, namely 1 - 0.653, and, yes, you would conclude that after seeing your experimental results, the probability of a fair coin is 0.347 and of an unfair coin is 0.653. Both priors (originally 0.1 and 0.9) must be updated. – user3697176 Apr 27 '15 at 04:38
  • @user3697176 Why must the priors be updated? Because of A difference, B significant difference or C other? – BCLC Aug 05 '15 at 21:01
  • 1
    C. You have additional information. At the start you believe that the coin is very likely to be fair, but then you find evidence (10 heads in a row) that does not occur very often if the coin truly is fair. This should shift your believes towards thinking the coin might be biased. – user3697176 Aug 05 '15 at 21:46
  • @user3697176 Thanks! Sorry wrong question. Is our need to update priors due to large difference between P(unfair) and P(unfair|evidence)? What if P(unfair|evidence) = 0.4? 0.2? 0.11? Also, what would updating prior mean? do we let P(unfair) now equal P(unfair|evidence)? – BCLC Aug 07 '15 at 16:47
  • 1
    @BCLC: The size of the difference has nothing to do with it. You update _every-time_ there is new evidence. The updated probability P(unfair|evidence) is usually called the posterior probability, and it can, of course, be used as a prior probability for the next round. – user3697176 Aug 09 '15 at 12:26
  • @user3697176 I may have misinterpreted you. When you said 'Both priors (originally 0.1 and 0.9) must be updated', I took it to mean that '*Hence* both priors (originally 0.1 and 0.9) must be updated' as in a particular course of action is dictated from the posterior probabilities or something related to it (eg the difference between posterior and prior probabilities). What did you mean by that exactly? Thanks! Also: [What exactly does it mean to and why must one update prior?](http://stats.stackexchange.com/questions/166321/what-exactly-does-it-mean-to-and-why-must-one-update-prior) – BCLC Aug 10 '15 at 04:23
  • Whenever you do a Bayesian analysis, you must update your prior distribution in the face of additional information. If the distribution is finite (as it is here --- two possible outcomes: fair (i.e., p(H) = 0.5) and unfair (p(H) = 2/3) this means updating the probabilities of all outcomes. To update the prior, you use Bayes' formula (see the original question). – user3697176 Aug 13 '15 at 13:05
  • @user3697176 '[Since we always update priors, the statement seems superfluous.](http://stats.stackexchange.com/questions/166321/what-exactly-does-it-mean-to-and-why-must-one-update-prior#comment316053_166322)' ? – BCLC Sep 07 '15 at 08:25