4

This question comes from a discussion on reddit.

One user asked a question:

Given $p=0.05$ in one study, what is the likelihood of $p \ge 0.05$ if the same study was repeated under the exact same condition in the same population?

With a set of possible answers: 5%, 10%, 50%, 95% and 99%.


The most common and most accepted answer seemed to be that you don't have enough information to tell.

However this seems wrong. My reasoning is the following:

0.05 is an arbitrary p-value. So the question really asks - when you obtain a p-value, what's the probability that the second p-value will be greater than the first one? And when rephrased that way 50% looks like an obvious answer.


The question is - what is the correct answer and how to derive it in a more rigorous way?

Chill2Macht
  • 5,639
  • 4
  • 25
  • 51
Karolis Koncevičius
  • 4,282
  • 7
  • 30
  • 47
  • 1
    This was extensively discussed in the linked thread. Note that there are several answers there and if you want to get the full picture of the debate, I would suggest to read all of them. I tried to provide some sort of non-technical overview/summary in my own answer there. Briefly, under certain assumptions the answer is indeed 50%, but without any assumptions the answer is "not enough information to tell". – amoeba Mar 19 '17 at 21:57
  • 1
    @amoeba Thank you for the link. On the first look it looks like it's spot on and more general than the current question. In that case it's possibly best to close this. – Karolis Koncevičius Mar 19 '17 at 22:03
  • 2
    Just a word of caution: my question there is about Cumming's paper that claims that the answer (50%) exists and does *not* depend on any assumptions. Cumming also posted an answer in that thread (presenting his argument) which is currently the most upvoted one. Nevertheless, the consensus of everybody else in that thread (including me) is that Cumming is wrong/misleading here. – amoeba Mar 19 '17 at 22:06

0 Answers0