2

As far as I understand it, the p-value is the probability of obtaining the observed data or more "extreme" values under the null hypotesis, concluding that if this probability is small enough the null hypotesis is likely false.

  1. Why are we interested in the probability of more "extreme" values as opposed to, for instance, the probability of some small interval around the obtained value? (i.e. the probability of obtaining a value in the interval [obtained_value - sigma/2, obtained_value + sigma/2] )

  2. What does "more extreme" actually mean? Sure, if we are talking of a test statistic following a normal distribution, "more extreme" can mean farther away from the mean. However, what about a bimodal distribution? Is there some formal way of stating what "more extreme" means?

Thank you!

Ciprian Talmacel
  • 21
  • 1
  • 3

1 Answers1

3

Traditionally you start NHST (Null Hypothesis Significance Testing) with a Null Hypothesis. Then you start collect data and try to reject that null hypothesis. The data you find either has reasonably likelihood unter the Null or is evidence against the Null Hypothesis. 'More extreme' is data, which has even less likelihood under the null hypothesis than some (not always so 'extreme') cutoff. If you think from the null hypothesis, it comes natural, that in the case of your hypothetical bimodal distribution, data in the middle of the two modes may be more extreme then the data on the right or the left of it. 'Extreme' in this sense does not necessarily mean large towards $\infty$ or small towards $-\infty$ but extreme with regard to a null hypothesis.

Bernhard
  • 7,419
  • 14
  • 36