0

As per statistical literature, there seems to be this understanding that given a p value less than alpha, we are to reject the null hypothesis.

The way I am interpreting this is that is the critical region, as defined by the alpha is the 'False Rejection' (TYPE 1 error) zone and constitutes the error threshold, why then p-value for a sample that is less than the alpha and within the error zone results in rejection of the null hypothesis?

Another line of thought - if the p-value is a way of saying how extreme a test statistic is for our sample data and seems to have a low probability, then how does it constitute evidence against the null hypothesis? Where am I going wrong?

Rajat Yadav
  • 1
  • 1
  • The p-value is calculated under the null model. Hence if the test statistic that was observed or a larger value has a low probability under the null model, then it suggests that the null model is unlikely to have been the model that generated the observed data. – hard2fathom Dec 06 '18 at 18:56

1 Answers1

0

The "alpha" ($\alpha$) is set a priori to be the probability of rejecting the null hypothesis when the null hypothesis is true (the Type I error as you stated). If, after conducting your statistical test, the $p$-value is found to be less than $\alpha$, this shows that, under the assumed null hypothesis, the probability you'd observe a test statistic as large (or larger) as the one you did, is very small. Something with such a small small probability of occuring should be considered suspicious. How could something be observed when it's so unlikely? Two things could have happened:

1) The assumption, under the null hypothesis is wrong (so you reject it/fail to accept it); or

2) You got a very unlucky result. This is simply the risk you are willing to take and you should set your $\alpha$/critical region accordingly when you set up your test. If you can tolerate more risk, set $\alpha$ to be larger. If you can't, set $\alpha$ to be smaller.

A. Donda
  • 2,819
  • 14
  • 32
StatsStudent
  • 10,205
  • 4
  • 37
  • 68
  • "likelihood you'd observe a test statistic as large" should be "probability you'd observe a test statistic as large". "likelihood" has a specific meaning and doesn't apply here. – A. Donda Dec 06 '18 at 19:54
  • 1
    That's a fair point, @A.Donda. I fell into the trap of interchanging everyday English with statistical terminology. The "likelihood" I mentioned above, should be taken to mean the everyday English version of it (i.e. probability). – StatsStudent Dec 06 '18 at 20:58