2

I compared the two distributions shown below using a Wilcoxon Rank Sum test (two-sided) using R. The p-value of the test was highly significant (as shown on the plot) and is based on 1,320 observations for each set

Although the overall distribution of Set 2 is shown to be lower than Set 1, the median of Set 2 is higher than the median of Set 1. Given this difference, how should the test results be interpreted?

Can it be stated that the values of Set 2 are significantly lower than those of Set 1? (even though the median of Set 2 is higher than the median of Set 1)

enter image description here

viridius
  • 41
  • 1
  • 1
  • 3

1 Answers1

3

Interpret the test in terms of what the test actually tells you.

The rank sum test (Wilcoxon-Mann-Whitney test, U test) considers whether $P(X>Y)$ differs from $\frac12$ (X being a random value from the first population and Y being a random value from the second, assuming continuity), and is effectively based on a sample equivalent of that (the proportion of cross-pair samples for which it is the case, compared to 1/2).

It doesn't compare medians; indeed medians may easily be in the opposite direction to the direction of difference the test indicates, because it's measuring differences in a different way. If you're making an additional assumption that would imply a difference in medians with a significant result (e.g. if you assume identical distributions under the null and a shift alternative -- which is sufficient but not necessary to imply a difference in medians) then you have grounds to doubt that additional assumption.

It's important to test the thing you want to know about! If you're interested in comparing medians, you might investigate whether you can do that (e.g. via a permutation test with an assumption of identical distributions under the null in order to have exchangeable group-labels; the alternative needn't require you to assume a shift alternative).

Glen_b
  • 257,508
  • 32
  • 553
  • 939
  • Thank you for the response. I am indeed wondering how to interpret these test results. Can it only be stated that Set 2 is different from Set 1? Or, is there a way to interpret the test results to indicate which Set generally contains larger values? – viridius Dec 14 '18 at 16:23
  • The precise interpretation is covered by my second paragraph. If you look at the specific form of statistic you're using (check your package - there's 3 or 4 variants in common use that are all simple functions of each other) and the value of the test statistic compared to its expected value under the null, which one tends to be larger should be clear. – Glen_b Dec 15 '18 at 01:06