4

I'm using a M-W U test to analyse some Likert scale results I have, as my data is ordinal.

One of the assumptions that keeps on appearing on references is the following: "both distributions must be the same shape (i.e., the distribution of scores for both categories of the independent variable must have the same shape).

If I fail this assumption, and therefore am unable to use M-W U, what can I use instead?

EDIT: I cannot add images, but will try to explain what my data looks like.

I have two groups based on my IV ("Yes" and "No"), charted on a boxplot with y-axis ranging from 1 - 5. For "Yes", I have a box extending from 4 to 5, with no variance. For "No", the box ranges from 3.5 to 4.5, with variance from 2.0 to 3.5, and 4.5 to 5.0.

I assume this means my shapes are "different". Does this mean I can't use MWU? If not, can you make suggestions as to what I can use instead (SPSS)?

EDIT2: Sample data:

"Yes": 5 5 5 5 5 5 4 4 4 4 5 5 4 4
"No":  4 5 5 5 3 2 4 3 4 4 4 4

I was using MWU to detect any significant differences (apparently, "yes") - but now am not sure if that's the right way to do it.

Gala
  • 8,323
  • 2
  • 28
  • 42
Stephen
  • 41
  • 1
  • 3
  • I posted similar question, it can be helpful. http://stats.stackexchange.com/questions/38881/how-to-test-group-differences-when-neither-parametric-nor-nonparametric-assumpti – sitems Jul 24 '13 at 12:34
  • Thanks - I saw it, but still don't really understand how to figure out the "shape" assumption, and what to do if it fails! – Stephen Jul 24 '13 at 12:37
  • 4
    The Wilcoxon test does not strictly make that assumption if you are careful in computing $P$-values. If you don't use the normal approximation but use a permutation exact test or use the full $U$-statistic method you can be OK. For the latter you can run the following in the R `Hmisc` package: `rcorr.cens(x, Surv(y))` and test $D_{xy}=0$ by getting the estimate for $D$ and its standard error. You can also use the generalization of the Wilcoxon test - the proportional odds ordinal logistic model. See the `orm` function in the `rms` package. – Frank Harrell Jul 24 '13 at 12:49
  • 2
    P.S. You can phrase the actual assumption made by the Wilcoxon test as being the proportional odds assumption, verifiable by plotting the logit transformation of the two ECDFs and checking for parallelism (linearity *not* required). – Frank Harrell Jul 24 '13 at 12:55
  • Thank you, Frank. I have to be honest - I'm not entirely sure what you mean by your answer; I use SPSS over R as I find it easier to understand. Is there any chance you could explain this a bit further so I can figure out how to replicate this in SPSS? – Stephen Jul 24 '13 at 13:05
  • You wrote "I have two groups based on my IV ("Yes" and "No"), charted on a boxplot with y-axis ranging from 1 - 5. For "Yes", I have a box extending from 4 to 5, with no variance. For "No", the box ranges from 3.5 to 4.5, with variance from 2.0 to 3.5, and 4.5 to 5.0." I don't think you mean "variance". Since the data go only 1 to 5, you could post a table. – Peter Flom Jul 24 '13 at 13:28
  • Regarding Frank's idea - I don't know SPSS, but surely it has ordinal logistic regression available. So, you can use that with the DV being the score and the IV being the group. Then you can test the proportional odds assumption. – Peter Flom Jul 24 '13 at 13:29
  • The boxplot description makes no sense. How many scores do you have? Can you include them in a table? That shape assumption does not mean identical and it doesn't sound like you've actually assessed it. – John Jul 24 '13 at 13:35
  • I doubt that SPSS can compute the exact P-value but see if it's an option. I am fairly certain that SPSS can fit the proportional odds model, and you can use the likelihood ratio $\chi^2$ test. – Frank Harrell Jul 24 '13 at 13:40
  • I have 26 scores ranging from 1 to 5, split across "Yes" and "No" (IV). Data now added above. – Stephen Jul 24 '13 at 13:40
  • @John: All the guides I've read indicate that to satisfy the shape assumption, you should create a boxplot and visually make sure that the "yes" and "no" shapes look similar. Is this wrong? If so, it would be great if you could give me pointers on how to actually assess this. I'm 99% sure MWU is the right tool, but find this assumption to be a stumbling block... – Stephen Jul 24 '13 at 16:27
  • 1
    You need to show that these two curves are parallel: the logit transform of the empirical cumulative distribution functions for group 1 vs. group 2. The curves do not need to be linear. – Frank Harrell Jul 24 '13 at 17:47
  • Figured out how to do this in SPSS, but my p < 0.05 now, which suggests that I've failed the assumption. Can I use alternative methods to statistically find differences between the two groups? – Stephen Jul 24 '13 at 20:53

2 Answers2

6

Here is an analysis using the R Hmisc and rms packages.

y1 <- c(5, 5, 5, 5, 5, 5, 4, 4, 4, 4, 5, 5, 4, 4)
y0 <- c(4, 5, 5, 5, 3, 2, 4, 3, 4, 4, 4, 4)

y <- c(y0, y1)
x <- c(rep(0, length(y0)), rep(1, length(y1)))

require(rms)
r <- rcorr.cens(x, Surv(y))
r

   C Index            Dxy           S.D.              n        missing 
 0.6773399      0.3546798      0.1378018     26.0000000      0.0000000 
uncensored Relevant Pairs     Concordant      Uncertain 
26.0000000    406.0000000    275.0000000      0.0000000 

z <- r['Dxy']/r['S.D.']
z   # 2.57


orm(y ~ x)

Logistic (Proportional Odds) Ordinal Regression Model

orm(formula = y ~ x)

Frequencies of Responses

 2  3  4  5 
 1  2 12 11 

                     Model Likelihood          Discrimination          Rank Discrim.    
                        Ratio Test                 Indexes                Indexes       
Obs            26    LR chi2      4.58    R2                  0.184    rho     0.407    
Unique Y        4    d.f.            1    g                   0.885                     
Median Y        4    Pr(> chi2) 0.0323    gr                  2.424                     
max |deriv| 2e-04    Score chi2   4.43    |Pr(Y>=median)-0.5| 0.381                     
                     Pr(> chi2) 0.0354                                                  

     Coef    S.E.   Wald Z Pr(>|Z|)
y>=3  2.6045 1.0536  2.47  0.0134  
y>=4  1.3459 0.6854  1.96  0.0496  
y>=5 -1.3459 0.6854 -1.96  0.0496  
x     1.7126 0.8426  2.03  0.0421

The likelihood ratio $\chi^{2}_{1}$ test for comparing the two groups yields $P=0.0323$

I obtained an exact $P$-value from the exactRanktests package wilcox.exact function of 0.04904.

Frank Harrell
  • 74,029
  • 5
  • 148
  • 322
  • Aha! I've figured out how to do this using SPSS (Analyse -> Regression -> Ordinal). I assume p < 0.05 implies that there's a significant difference between my two groups and that I can NOT use M-W U for analysis as I've failed the assumption. Is this right? Can you help me identify alternative methods I can use? – Stephen Jul 24 '13 at 20:47
  • 1
    No I wouldn't say that. First of all, get away from the magic 0.05 cutoff. Second, you haven't shown anything indicating that assumptions (proportional odds) don't hold. The proportional odds model is an excellent way to handle heavy ties in the data, and it allows adjustment for other variables. The proportional odds model and the Wilcoxon test have the same assumptions if you compute $P$-values optimally. BTW your sample size is too small to really examine the assumptions. – Frank Harrell Jul 24 '13 at 21:15
  • So can I say something along the lines of, "The sample size is too small to determine whether the assumption holds" and run the M-W U anyway? I'm happy with the stats I have; just not sure how to answer a possible question about the assumption that "both distributions must be the same shape"... – Stephen Jul 24 '13 at 21:18
  • 2
    Yes, or say it more positively: The Wilcoxon-Mann-Whitney 2-sample test is among the most robust of tests. Having fewer assumptions than parametric tests, the Wilcoxon test is less likely than the $t$-test to be affected by assumptions not holding. The Wilcoxon test doesn't assume that the two distributions have the same shape; it assumes that the two distributions, whatever they are, are connected by the proportional odds assumption. – Frank Harrell Jul 24 '13 at 21:43
  • Wonderful. *Thank you* so much for your help. Can you suggest a reference, in particular to the "less likely than the t-test to be affected by assumptions not holding"? – Stephen Jul 24 '13 at 21:49
  • Virtually any textbook on nonparametric statistics. – Frank Harrell Jul 24 '13 at 22:38
  • 1
    @FrankHarrell It depends on which assumptions we're considering. To my recollection, it seems to be slightly more affected that the t-test by serial correlation, for example. However, overall it's both quite robust overall and against the assumptions people tend to worry about, generally more robust than the t-test. – Glen_b Jul 24 '13 at 23:33
1

The rank sum test does not strictly speaking have a shape assumption. At the most general, the rank sum test is a test of stochastic dominance with a null hypothesis H$_{0} \text{: P}(X_{A} > X_{B}) = \frac{1}{2}$; i.e. the probability of observing a randomly selected value from group A that is larger than a randomly selected value from group B is one half). With two additional assumptions, the rank sum test can be interpreted as a test for median difference with a null hypothesis H$_{0} \text{: } \tilde{x}_{a} = \tilde{x}_{b}$. These two additional assumptions are:

  1. That the distributions have the same shape, and
  2. That any differences in these distributions are differences in central location.

Therefore distributional differences between groups do not invalidate the use of the rank sum test, but rather alter the interpretation of its results.

See also my response here: ANOVA / Kruskal-Wallis: one of four groups has different distribution.

Community
  • 1
Alexis
  • 26,219
  • 5
  • 78
  • 131