8

I want to determine if there's a difference in mean p-values between two groups. In order to do this I perform a Wilcoxon's rank-sum test (the data is not normally distributed). So far, so good. Finally, I want to calculate the corresponding effect size. Unfortunately, R does not provide this. It also does not provide a z value with which the effect size can easily be calculated using: effect size = z / sqrt(N)

here is some sample R code:

a=rep(0:1,each=20)                            #grouping variable
b=c(rnorm(20, .03,.01), rnorm(20, .02, .009)) #vector of p-values

d=cbind(a,b)

test = wilcox.test(b ~ a, data = d)     #perform Wilcoxon rank-sum test
test

Does anybody know how to obtain the effect size?

amoeba
  • 93,463
  • 28
  • 275
  • 317
mats
  • 81
  • 1
  • 1
  • 2
  • 3
    Welcome to SO. I flagged your question for migration to www.crossvalidated.com , as your question is more statistical than anything else. In short : Wilcoxon RANK test works with ranks, so I'm not sure about which effect size you're talking. Obviously it doesn't give you a z value, as that one is linked to parametric testing, not to non-parametric tests like Wilcoxon. Wilcoxon has to be interpreted in terms of location shift. – Joris Meys Jun 14 '12 at 14:47
  • 6
    The [Wilcoxon rank sum test](http://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U) does not evaluate a difference in *means*: it tests for stochastic inequality. The proper measure of an effect size therefore is the probability that $B \gt A$, minus the null value of $1/2$. – whuber Jun 14 '12 at 15:07
  • @whuber And how is this effect size calculated? From what I've read, it must be a number between 0 and 1 (or between -1 and 1?), but the most voted answer provides a different number (-6 in my case, even divided by sqrt(N) it's still < -1). – Rodrigo Nov 13 '20 at 10:30
  • Rodrigo, please see https://stats.stackexchange.com/questions/133077/effect-size-to-wilcoxon-signed-rank-test and https://stats.stackexchange.com/questions/342987/effect-size-for-wilcoxon-signed-rank-test-that-incorporates-the-possible-range-o. – whuber Nov 13 '20 at 14:23

2 Answers2

13

The estimator that corresponds to the Wilcoxon test is the Hodges-Lehmann estimator; it's returned by wilcox.test using the conf.int=TRUE option, under "difference in location".

For your example:

> wilcox.test(b~a,data=d, conf.int=TRUE)

    Wilcoxon rank sum test

data:  b by a 
W = 355, p-value = 6.914e-06
alternative hypothesis: true location shift is not equal to 0 
95 percent confidence interval:
 0.008657301 0.021523993 
sample estimates:
difference in location 
            0.01442617

For more on the Wilcoxon and the assumptions behind it, and what it actually tests for, and other nonparametric estimators, this document is (possibly) helpful: www.stat.umn.edu/geyer/old03/5102/notes/rank.pdf

jbowman
  • 31,550
  • 8
  • 54
  • 107
  • So what you say is that the effect size is `wilcox.test(b~a,data=d, conf.int=TRUE)$estimate / sqrt(20)`, correct? –  Jan 09 '15 at 13:29
2

Get the z for your formula by

library(coin)
mydf <- as.data.frame(d)
wilcoxsign_test(b ~ a, data = mydf, distribution="exact")

and compute the effect size with your formula, setting N to 40

Primigenius
  • 21
  • 1
  • 1
    I suspect this reply would be of higher interest if you could supply additional details on the reasons to use the `coin::wilcoxsign_test` R function. Also, are you referring to the OP formula, $\text{effect size} = z / \sqrt{N}$? – chl Apr 04 '13 at 19:52
  • The reason ti use coin::wilcoxsign_test is that it computes a z value. – Primigenius Apr 05 '13 at 09:43
  • 1
    Another way to obtain z from standard wilcox.test is the algorithm Andy Fields implemented in his function [rFromWilcox](http://www.sagepub.com/dsur/study/DSUR%20R%20Script%20Files/Chapter%2015%20DSUR%20Nonparametric.R). And yes, I was referring to effect size = z / sqrt(N). – Primigenius Apr 05 '13 at 18:17
  • 2
    Thanks for the clarification although I'm still unsure about that z-value being a proper measure of effect size. (+1) – chl Apr 05 '13 at 20:47