14

These two functions exist in R but I don't know their differences. It seems that they only return the same p-values when calling wilcox.test with correct=FALSE, and wilcox_test (in the coin package) with distribution="aymptotic". For other values they return different p-values. Also wilcox.test is always returning W=0 for my dataset, independently of the settings of its parameters:

x = c(1, 1, 1, 3, 3, 3, 3) and y = c(4, 4, 6, 7, 7, 8, 10)

Also, when I try using different tools other than R (some available online, others as Excel add-ons), sometimes they report different p-values.

So how can I know which tool is giving the "correct" p-value?

Is there a "correct" p-value, or if a few tools give a p-value < 0.05 should I be happy? (Sometimes these tools do not offer so many parametrization possibilities like R.)

What am I missing here?

amoeba
  • 93,463
  • 28
  • 275
  • 317
mljrg
  • 551
  • 1
  • 7
  • 13

1 Answers1

18

The key to your question is found in ?wilcox.test in the Notes section:

The literature is not unanimous about the definitions of the Wilcoxon rank sum and Mann-Whitney tests.

So what that means is there is more than one way to do this non-parametric test of change in location between two samples. In addition, given each definition, there is more than one way to get a p value. "exact" means that it is absolutely correct, while "approximate" or "asymptotic" are both approximations of the truth. That's why there are multiple options in both wilcox.test() and wilcox_test(), and only some of them match exactly - when you have both functions doing exactly the same thing. It looks like wilcox_test() can get exact p-values even when there are tied values, while wilcox.test() falls back to an asymptotic approximation when there are tied values. I wouldn't know what combination of statistic and p-value calculations an Excel add-on is doing, but the advantage of R is that it is clear what choices you have made.

Your next question is why wilcox.test() is returning 0 all the time. For the data set you created, the value of the test statistic is 0 when you do wilcox.test(x,y) but it will be 49 when you do wilcox.test(y,x) although the p-value will be the same. See the wikipedia page for the reasons. wilcox_test() returns a Z transformation of the statistic returned by wilcox.test(), which is why they have different values of the test statistic.

Is there a correct p-value? Yes, but sometimes it is too hard to calculate, and so we need to use approximate methods (see ?wilcox_test for descriptions of how the exact calculation can fail from insufficient memory). The differences between approximation and exact value will mostly not matter unless the true difference in the location of the two groups is very small.

atiretoo
  • 1,458
  • 14
  • 29