5

The interpretation of the p-value is a very difficult thing because what is legit and what is not is very close together. The p-value is the probability of the resulting test statistic if the null hypothesis $H_0$ is true; that means $\Pr(\mathrm{data}|H_0)$. A lot of people make the mistake of confusing this with $\Pr(H_0|\mathrm{data})$, which is wrong. Now to my question: I stumbled upon an online independent t-test and the result was a p-value of 0.20. The given interpretation was the following: "There is a 79.76% chance the proportions are different."

My question: is this interpretation legit? Or is it the same confusion as mentioned above (e.g. it describes the probability of $H_0$). Thanks in advance!

Scortchi - Reinstate Monica
  • 27,560
  • 8
  • 81
  • 248
duplex
  • 79
  • 1
  • 2
  • 4
    _"the p-value is the probability of the resulting test statistic if h0 is true"_. That's not correct. _"[...] the p-value is the probability of obtaining a test statistic at least as extreme as the one that was actually observed, assuming that the null hypothesis is true"._ (from Wikipedia) – boscovich May 31 '13 at 13:45
  • 3
    This interpretation is **wrong.** As you said: the $p$-value is the probability of observing a $t$-value at least as extreme as the one you found *if the null hypothesis were true.* – COOLSerdash May 31 '13 at 13:46
  • Because the test was already performed, the conclusion of whether the means are different or not different should not have a probability attached to it. You either reject, or not reject, cannot 20.24% reject. – Penguin_Knight May 31 '13 at 13:47

1 Answers1

9

The null hypothesis in this case is that the proportions are equal. One alternative hypothesis that the two are not equal. The statement "there is a 79.76% chance that that the proportions are different" is a statement about the probability of the alternative hypothesis. This is not the correct interpretation of the p-value. Recall that a p-value is the probability that, purely due to the features of random sampling under the null model, you would observe a test statistic at least as extreme as that calculated for your particular sample. All the p-value tells you is the probability of observing the data, given the null model.

Scortchi - Reinstate Monica
  • 27,560
  • 8
  • 81
  • 248
Sycorax
  • 76,417
  • 20
  • 189
  • 313
  • 3
    Close, but you're forgetting about the test statistic, which partitions the sample space & orders those partitions: "...under the null model you would observe a test statistic at least as extreme as that calculated for your particular sample" would do it. – Scortchi - Reinstate Monica May 31 '13 at 14:00