3

I have three groups. Both have small sample sizes (<10). And we can't check the assumption of normality. I want to compare any two of them.

What to do in this case( unequal small sizes, non-normality)

OMG
  • 171
  • 4
  • How about a permutation test? You may not have much here with such small samples, but on the plus side you can do full enumeration. – soakley Dec 07 '13 at 23:27
  • I don't understand. How to carry out a permutation test ？ What's its assumptions? – OMG Dec 07 '13 at 23:32
  • So much I want to say about this, but first some questions -- when you say 'I want to compare them' - what is it you're trying to compare? Do you mean to compare locations? Is there a specific alternative? Are you talking about 3 pairwise comparisons or are you after something notionally more like a one-way ANOVA? (The suitability of permutation tests will be affected by how small the sample sizes are -- if they're small enough, you won't be able to get below some specific significance levels; on the other hand, outside specific parametric assumptions you may not have much choice about that) – Glen_b Dec 07 '13 at 23:55
  • A good description of a permutation test will not fit in a comment. But check Wikipedia under "Resampling" for a decent start. – soakley Dec 08 '13 at 00:13
  • Actually I did anova and rejected the null hypothesis. So I want to know the pair comparison results. – OMG Dec 08 '13 at 00:37
  • 1
    So does that mean you want to change your question to 'how do I do multiple comparisons after an ANOVA?' or do you actually want to know something like 'how do I do something like an ANOVA and multiple comparisons but without assuming normality?' ...? (Incidentally, what are your sample sizes?) – Glen_b Dec 08 '13 at 01:13

2 Answers2

2

(This is too long for a comment; if the question is sufficiently clarified - see the comments - it may be edited into a better answer.)

On permutation tests:

The basic idea of a permutation test is that if the null hypothesis is true, the group/category labels are arbitrary and could be reassigned without changing the distribution of the groups. So they are reassigned, in every fashion and a test statistic sensitive to the alternative is calculated; if the null is true, the sample should be a randomly chosen member of that permutation distribution. If the alternative is true, the sample should tend toward being 'extreme' in that distribution (extreme in a sense defined by the ordering resulting from what is most consistent with the alternative).

There's some discussion of permutation tests here, discussion of why they might be used here, discussion of the relationship between permutation tests and randomization tests here, and a little on the choice between permutation tests and t-tests here.

There are examples of different permutation tests done here and here and here and here (see near the end).

Community
  • 1
Glen_b
  • 257,508
  • 32
  • 553
  • 939
0

No matter what you use your power to detect differences will be low. Another possible approach would be using non-parametric tests. If you have three groups you could use Kruskal-Wallis (followed by a multiple comparison procedure if you want to compare them pairwise).

In R you can use kruskal.test for the Kruskall Wallis test and kruskalmc in the pgirmess package for multiple comparisons

Aghila
  • 657
  • 5
  • 18