1

I've got to teach a basic course on the $\chi^2$ test soon, and I will of course insist on the fact that $p$ is not the probability of the null hypothesis to be true, and that the result of the test can't allow to accept the null. However, students of this course won't have any mathematical or statistical background, and I'd like to provide an explanation as intuitive as possible.

I've read some interesting things in this ScienceNews article, with an example of a dog barking when he's hungry, but I still don't find this really clear enough.

Did you already provide such an explanation, or have any idea or example that would be "sufficiently intuitive" ?

Thanks a lot in advance.

juba
  • 255
  • 3
  • 9
  • Already: [here](http://stats.stackexchange.com/questions/31/what-is-the-meaning-of-p-values-and-t-values-in-statistical-tests), [here](http://stats.stackexchange.com/questions/5591/why-do-people-use-p-values), [here](http://stats.stackexchange.com/questions/16939/why-is-it-bad-to-teach-students-that-p-values-are-the-probability-that-findings), & probably elsewhere. – Scortchi - Reinstate Monica Jul 03 '13 at 17:28
  • 1
    I think there are a couple of threads on CV that might help you; take a look at these: [Why does a 95% CI not imply a 95% chance of containing the mean?](http://stats.stackexchange.com/questions/26450/), [Understanding p-value](http://stats.stackexchange.com/questions/44769/), & [Interpretation of p-value in hypothesis testing](http://stats.stackexchange.com/questions/46856/). – gung - Reinstate Monica Jul 03 '13 at 17:30
  • I don't think there will be a good intuitive explanation of why the p-value is not the probability that the null hypothesis is true because the principal reason this isn't true (that frequentist statistics cannot assign a probability to the truth of a hypothesis as it has no valid interpretation as a long run frequency) is deeply counter-intuitive. – Dikran Marsupial Jul 03 '13 at 17:46
  • For teaching purposes, I would point out that we often use a null hypothesis that we know from the outset is false (e.g. the coin is exactly unbiased), in which case a non-zero p-value cannot possibly be the probability that the false null hypothesis is true! – Dikran Marsupial Jul 03 '13 at 17:47
  • 2
    For what it's worth, I don't think this is an exact duplicate. You might try editing this in a way to make 'looking for teaching strategies for making the fact that $p(d|H_0)\ne p(H_0|d)$ intuitive' more distinct from the prior questions, & maybe it can be reopened. – gung - Reinstate Monica Jul 03 '13 at 18:20
  • @gung How would such an edit make this question differ in any way from http://stats.stackexchange.com/questions/31, which already asks for the same explanation in a classroom context and already has at least one answer specifically addressing the "$p \ne$ probability of null" issue? – whuber Jul 03 '13 at 18:57
  • @whuber, it's true that the question includes "[h]ow would you explain the following points to college students...", but the thread doesn't seem to me to center on *teaching strategies for making this intuitive* in the sense that, eg, I think this thread: [strategies-for-teaching-the-sampling-distribution](http://stats.stackexchange.com/questions/34926/), does. It seems to me that there is room for such a version, but I acknowledge that it's just my opinion & I abide by the community's decision. – gung - Reinstate Monica Jul 03 '13 at 19:06
  • this question is a lot more specific than the previous question, which makes it easier to find the specific answer to a common misconception. – Dikran Marsupial Jul 03 '13 at 19:10
  • 1
    @gung I would add my vote to reopen if the focus were on *teaching strategies* (instead of "explanation") and specifically on the "$p\ne$ probability of null" question, because those elements could lead to a different set of answers. – whuber Jul 03 '13 at 19:11
  • First, thanks for all your comments, which are very helpful. I don't think I will edit my question and ask for a reopening, because after reading the links you give I think it was flawed. The correct title should have been *How to explain students intuitively that p is not a probability related to either hypothesis in a test*. – juba Jul 03 '13 at 20:43

1 Answers1

3

Here is an example that I use for teaching. I take a real coin and toss it three times. Every time I get the same result. We go through the calculations: we find that the probability that we get three tails in three throws under H0 is $\frac{1}{8} = 0.125 > 0.05$. Thus, we cannot reject the H0.

Then I continue to toss the coin, and every time I am getting exactly the same result. The students (without doing any calculations) see that the coin (in fact, my tossing procedure) is rigged. We reject the H0. The H0 never was true; we just did not have enough data to demonstrate that. Also, we see that the calculations of the p value only involve the parameter for a fair coin and a fair toss.

I also use the same example to demonstrate a few other pitfalls. For example, the students usually assume that the coin is rigged; the fact that it is not serves as a metaphor of a flawed measurement procedure.

Here is how to get the same result every time: http://www.youtube.com/watch?v=jQOX7Gwz69U

(note that I do not teach statistics for living; rather, it is a part of a bioinformatic course)

January
  • 6,999
  • 1
  • 32
  • 55
  • What about the test statistic? After all the probability under the null of heads, followed by tails, then by heads - indeed of any sequence - is also 0.125 – Scortchi - Reinstate Monica Jul 03 '13 at 17:40
  • You are right, I should have not written "consecutive" – January Jul 03 '13 at 18:44
  • I do not want to twist this answer, but it seems that the demonstration could be misconstrued as showing that *asymptotically,* a sufficiently large experiment will produce a p-value that is *approximately* the same as the "probability of $H_0$." In other words, this (nice) demonstration might not accomplish the learning objectives you believe it does, because it does not appear directly to attack the fundamental misconception (in the frequentist paradigm, of course) that it even makes sense to assign a probability to the null. – whuber Jul 03 '13 at 19:14