2

I am running a large number of two-sample Wilcox tests and using Benjamini, Hochberg, and Yekutieli method to adjust the p-values.

The issue is that when I conduct a one-tailed test, the p-values before the adjustment satisfy p-value(>) + p-value(<) = 1, which makes sense. After the adjustment, however, this relation is not satisfied any more, and on top of that, in some tests, the alternative hypothesis is accepted in both directions. That is the p-value(X>Y) > 0.95 and p-value(Y>X) > 0.95 ! (Both are one-tailed tests)

So my question is: Is it valid to adjust p-values of one-tailed tests? If yes, how to make sense of the result?

Edit: Sorry, I cannot comment yet since I do not have the necessary reputation. The reason I am running both directions is: first, to make sure that the adjustment makes sense (which does not seem to be the case as far as I can tell). The second reason is to have a clear display of the results: I want a matrix where I can read columns-wise or row-wise and see which sample is significantly smaller or larger than the other. A two-tailed test will only tell whether the means are different, and I have to check the value of the means in another table to see which sample is actually smaller.

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
S.K.
  • 21
  • 1
  • Are you testing BOTH whether X>Y and whether Y>X? If so, why not use a two-tailed test instead? – MånsT Jan 10 '18 at 09:05
  • why do you want to adjust p- values ? And does pv stands for p-value. –  Jan 10 '18 at 09:14
  • what is your hypothesis that you are planing to test through two sample tests that you are conducting right now ? –  Jan 10 '18 at 11:17
  • That is p-value(X>Y) > 0.95 and p-value(Y>X) > 0.95 ? what do you mean ? Does it mean both directions or two tailed ? –  Jan 10 '18 at 11:22
  • p-value(>) + p-value( –  Jan 11 '18 at 11:21

0 Answers0