0

I am reading a paper that is using permutations to compare the means of two different treatment groups (a nutrition study, took minimum - maximum) that have low sample sizes, and so the groups are not normal, despite any transformation. Makes sense to me.

However, the table just reports p values. I'm not really familiar with pairwise permutation tests, but I was expecting some kind of effect size. Am I wrong, or can anyone point me in the right direction?

Michael
  • 13
  • 2
  • An effect size would definitely be better, & should be possible. However, it's (unfortunately) common for effect sizes not to be reported in papers. Can you contact the corresponding author & ask for them? – gung - Reinstate Monica Feb 27 '22 at 22:18
  • that's the idea, im just wondeirng more about the methods, i.e. the permutaions, and whether it precludes an effect size calculation. They have low sample size (n=6) so maybe they figured an effect size wouldn't be that telling – Michael Feb 27 '22 at 22:52

1 Answers1

0

Using a permutation test doesn't necessarily preclude an effect size, although the connection between the test and the effect size may be broken. In general, a permutation test works by reshuffling the data and computing a statistic many times. The statistic could be something like a mean difference, or it could be a test statistic (e.g., $t$). Either way, this allows for an empirical estimate of the sampling distribution under the null. Comparing your statistic to the sampling distribution allows you to compute a $p$-value. A simple example of a permutation test can be seen in @jbowman's answer here: The z-test vs the χ2-test for comparing the odds of catching a cold in 2 groups. For a pairwise variant, the shuffling would only be within the pairs, but otherwise the principle is the same.

gung - Reinstate Monica
  • 132,789
  • 81
  • 357
  • 650