Wikipedia says that p-values are uniformly distributed over [0,1] if the null hypothesis is true and for continuous data.
What is the expected p-value distribution if the test statistic is discrete? E.g. occurrences in an enrichment analysis.
Asked
Active
Viewed 1,178 times
1
-
3See the discussion at [Non-uniform distribution of p-values when simulating binomial tests under the null hypothesis](https://stats.stackexchange.com/a/153267/805) -- a step function that touches the $y=x$ line at the top of each step is the general case. I suggest there that it could be called `quasi-uniform` while whuber has argued in comments elsewhere on site that is should just be called 'discrete uniform'. However, that doesn't match a fairly [conventional usage of the term](https://en.wikipedia.org/wiki/Discrete_uniform_distribution), so I don't think that suggestion will catch on. – Glen_b Oct 19 '17 at 09:20
-
2Note that it's not the data but the distribution of the test statistic that is at issue. Consider, for example, a two-sample Kolmogorov-Smirnov test; the data can be continuous but the discreteness of the distribution of the test statistic is the issue. I have edited the question accordingly. – Glen_b Oct 19 '17 at 09:27
-
1Another (very common) situation in which the Wikipedia claim is false is a "composite null" hypothesis, such as testing $H_0:\mu \le 0$ against the alternative $H_A:\mu \gt 0$ for data from a Normal$(\mu,\sigma^2)$ distribution. The p-value will be uniformly distributed only when $\mu=0$; if $\mu\lt 0$ (which is still part of the null hypothesis), the p-value will be concentrated closer to $1$. – whuber Oct 19 '17 at 14:22