0

So I have a small sample size (N=24) but found an extremely large effect size for an effect. A common phrase appearing is 'effect size is independent of sample size,' which I had taken to suggest that although my sample size was small, my large effect size suggests a strong effect is present.

However, I have found other research suggesting a small sample size can inflate effect size.

Therefore my question is whether it is appropriate to conclude that my strong effect size alongside significant effect with the small sample size, still suggests there is a strong effect present?

Thanks!

Holly Miller
  • 33
  • 7

1 Answers1

2

The issues with small samples is that statistics are not precise and the sample may not be representative of other groups. Ideally, you would collect more data. If you can't do that, I suggest you calculate confidence intervals around your effect size to see how precise/imprecise it is. However, keep in mind bootstrapping is not a cure for small sample sizes.

In terms of how strongly you feel that effect size is accurate, you likely want to be cautious about generalizing the size of the effect to others. How strongly you want to rely on results from a small sample is in part related to what the prior literature has found. If the literature on this effect states it commonly has a large effect size in other groups, I think it may be safe to state that the effect also appeared large in your sample and is in line with the literature. If the literature has no information or finds small to moderate effect sizes, I would only discuss the size of the effect as an indication that you need more data for verification.

Mark
  • 325
  • 1
  • 6
  • Thank you that helps! I used partial eta-squared to calculate my effect sizes (two way repeated measures ANOVA), but have read some comments saying omega squared could be more useful for smaller samples. Do you think calculating this effect size alongside CI would be more beneficial, or to stick with partial eta squared? – Holly Miller Mar 11 '19 at 16:11