5

Testing the difference between the observations of two groups by using Mann-Whitney Test has given the following output (from minitab):

                N  Median
positives  137  1.0000
negatives  892  1.0000
Point estimate for η1 - η2 is 0.0000
99.0 Percent CI for η1 - η2 is (0.0001,-0.0001)
W = 56899.5
Test of η1 = η2 vs η1 ≠ η2 is significant at 0.0000
The test is significant at 0.0000 (adjusted for ties)

When I was testing other pairs of groups, I concluded that a group tends to have larger values than another group based on their medians, if the test shows a statistically significant difference. However, the example above shows that I may not be interpreting the results correctly.

Here is some additional descriptive statistics for the example above:

Variable   |  Mean | StDev  | Minimum | Q1 |Median| Q3 | Maximum
positives  |  4.13 | 13.17  |  1.00   |1.00| 1.00 |1.00| 116.00
negatives  | 6.851 | 20.503 |  0.000  |1.00| 1.00 |5.00| 434.000
  • What have I done wrong?
  • How can I figure out the direction of the difference? Can I conclude that the negatives tends to have larger values based on their larger Q3, for example?
PatternRecognition
  • 623
  • 5
  • 19

2 Answers2

6

The Mann-Whitney is not a test of medians. At best, the Mann-Whitney test can only be claimed to a be a test of differences in mean-rank between two populations' pooled ranking.

You can easily calculate medians empirically and perform a basic Wald test if you need a test of medians.

The Mann-Whitney test happens to be a reasonably powerful test of medians only when the underlying distributions are symmetric, an assumption that is clearly violated in these data. However, if a distribution is symmetric, the median also happens to be the mean (when variance is finite). This means the Mann-Whitney and the t-test are testing the same hypothesis in symmetric distributions.

AdamO
  • 52,330
  • 5
  • 104
  • 209
  • Thanks. I do not particularly care about the medians, all what I care about is (1) the significance and (2) the direction of the difference. How can I tell the difference direction? – PatternRecognition Dec 30 '15 at 22:34
  • @patternrecognition use a ttest – AdamO Dec 30 '15 at 22:40
  • excuse my ignorance, could you please give me more details? – PatternRecognition Dec 30 '15 at 22:49
  • 1
    (+1) ... "This means the Mann-Whitney and the t-test are testing the same hypothesis in symmetric distributions." ... for which the mean exists. If you had data from Cauchy random variables (say), then you couldn't really say you were testing for equality of medians. – Glen_b Dec 30 '15 at 23:25
  • Thanks. I did not want to know how to fit a t-test. I however wanted to know more about how the test itself can meet my needs. Any way, I will try to figure this out. – PatternRecognition Dec 31 '15 at 19:37
  • 1
    @PatternRecognition I see. Then it's simply a matter of stating a clear hypothesis. The mean is a *very* natural measure to compare between two populations, it's almost expected by anyone in science or business. In any case where Mann Whitney is used, I almost always ask myself why they did not use a t-test: the t-test has measures of direction (the mean difference) as well as significance (p-value). Both can be presented very nicely with a 95% confidence interval. – AdamO Dec 31 '15 at 19:45
  • Thanks. My data is not normally distributed and, so, can I still use the t-test? Please excuse my ignorance, I have a very basic understanding of statistics. – PatternRecognition Dec 31 '15 at 20:13
  • 1
    @PatternRecognition with a sample size of over 1,000 you can be assured that there are no assumptions of normality necessary for the t-test. Basically, due to the central limit theorem, the sampling distribution of the test statistic will undoubtedly tend very closely toward a normal distribution anyway. – AdamO Dec 31 '15 at 21:58
  • Which test to use instead of Mann-Whitney test when the underlying distributions are not symmetric? – vasili111 Jan 09 '20 at 15:06
  • @vasili111 It will depend on a number of factors. If you have a large sample size like 1000 mentioned previously, t-testing is likely to be fine. Do you have a particular situation in mind? And what do you want to test? – Dave Jan 09 '20 at 16:16
  • @Dave I may need to test 2 independent samples that have non-parametric continues data and I want to be prepared for it. I do not have details other than sample size is ~40 and it is most likely non-parametric data where I will need to compare 2 independent samples with continues data. I know that for non-parametric dependent samples Wilcox sign rank test is good if there is a symmetry if not it is advised to use sing test. What about two independent continues non-parametric data that has no symmetry? Which test is advised in that case? – vasili111 Jan 09 '20 at 16:29
  • @vasili111 What is non-parametric data? – Dave Jan 09 '20 at 16:35
  • @Dave not normally distributed. – vasili111 Jan 09 '20 at 16:45
  • @vasili111 Consider how your data are distributed, search through previous Cross Validated questions to see if your question has been addressed, and write a new post if you remain uncertain. – Dave Jan 09 '20 at 17:00
1

Mann-Whitney U test is a rank-sum test, hence it doesn't really care about distribution properties such as mean, media, etc, it only cares that one of your variables tends to have higher values than the other, hence the former has a higher sum of ranks. Nevertheless, if you look closely at this table: 

Variable   |  Mean | StDev  | Minimum | Q1 |Median| Q3 | Maximum
positives  |  4.13 | 13.17  |  1.00   |1.00| 1.00 |1.00| 116.00
negatives  | 6.851 | 20.503 |  0.000  |1.00| 1.00 |5.00| 434.000

you might notice that both variables have equal 25-percentiles, means and medians, while the seconds one has a higher 75-percentile. That supports the observation that the second distribution is likely to have a higher rank-sum.

Edit

Inspired by AdamO's comment, I made a little research on the U-test. According to this published response in Arthritis and Rheumatism journal (impact-factor >7) the test can only compare two distributions of similar shape. This assumption is clearly violated.

The Mann-Whitney U test (2) and the Kruskal-Wallis test (3) are nonparametric methods designed to detect whether 2 or more samples come from the same distribution or to test whether medians between comparison groups are different, under the assumption that the shapes of the underlying distributions are the same. Thus, these nonparametric tests are commonly used to determine whether medians, not means, are different between comparison groups. Although these tests are often used to compare means when normality assumption is not violated, strictly speaking, interpreting the results of non- parametric tests for mean comparison is inaccurate. When the distribution of a variable is skewed (for example, as in the values for C-reactive protein that van der Helm-van Mil et al present in Table 2 of their article 1), only assertions on whether medians, and not means, were different between groups should be made using nonparametric methods.

Given that your sample is not small I would recommend you to try a permutation test to answer your question. Here is a good discussion on their limitations

Community
  • 1
Eli Korvigo
  • 771
  • 1
  • 6
  • 16
  • 2
    You are confusing "parameters" with distributional properties. All continuous distributions have a median. The Mann-Whitney test is no less "non-parametric" than the t-test in that regard, except in the rare cases when a distribution doesn't have a mean. One can always make a test for differences in means or rank means regardless of the distribution, and it is the limiting distributions of their test statistics which tend to Gaussian, rather than exact probabilistic calculations on which inference is based. – AdamO Dec 30 '15 at 22:21
  • @AdamO wow, it's the first time someone tells me that U-test assumes symmetrical distributions. As far as I remember it, being nonparametric, doesn't have this limitation. In fact, the only limitation I can think about is some huge difference in variance. – Eli Korvigo Dec 30 '15 at 22:27
  • @EliKorvigo Can I say that the second distribution *does* have a higher rank-sum? [OR] should I say it is *likely* to have a higher rank-sum ? – PatternRecognition Dec 30 '15 at 22:39
  • @PatternRecognition see my edit – Eli Korvigo Dec 30 '15 at 23:24
  • @EliKorvigo I don't think AdamO said that the U-test assumes symmetrical distributions. – Glen_b Dec 30 '15 at 23:27
  • @Glen_b An excerpt from AdamO's answer "The Mann-Whitney test happens to be a reasonably powerful test of medians only when the underlying distributions are symmetric" – Eli Korvigo Dec 30 '15 at 23:30
  • 2
    Note first that the Mann-Whitney test is NOT a test for equality of medians. The sentence you quote doesn't mean that Mann-Whitney test assumes symmetry. Instead it explains that if you *also* assume symmetry (which is not normally an assumption of the test), *then* it becomes a test for equality of medians. That is, by adding an additional assumption to the test, you can have it test medians -- but as he says, then it also tests means (at least when population means exist) – Glen_b Dec 30 '15 at 23:38