1

I am comparing measures of a DV x with two group factors (2-way ANOVA) - y (with 2 levels) and k (with 3 levels). Both present significant main effects (p<.001) and high partial eta squared (above 0.6). The interaction effect is also significant (p<0.001) but the partial eta squared is 0.029. Post-hoc testing for the interaction of y*k yields significant differences on the DV for all comparisons. In this case, shouldn't the effect size be greater? How do I interpret this? Thank you for your help!

P.S. - Note: (I have a big sample; N>600)

Pedro Silva
  • 11
  • 3

2 Answers2

4

At large enough sample sizes, any effect will be significant, even miniscule ones. You interpret it as a small effect that is nevertheless statistically significant (that is, large enough to be distinguished from random variation). That doesn't mean it's important or relevant.

Glen_b
  • 257,508
  • 32
  • 553
  • 939
  • Thank you! Given the small effect size, do you think I should report post-hoc? All comparisons are significant, but it might be due to the sample size as well. –  Mar 19 '13 at 11:08
  • @Pedro, it looks like your forgot your login information. Please visit our [help page](http://stats.stackexchange.com/help/user-merge) to merge your two accounts and don't forget to register once and for all. – chl Mar 19 '13 at 11:23
  • Done with login! – Pedro Silva Mar 19 '13 at 16:28
  • I almost upvoted, but if we're going for "canonical answers" (not that many of mine are), either the "large enough" or the footnote will have to go in order for the answer to make sense. – rolando2 Mar 20 '13 at 00:40
  • Yeah, I bothered me somewhat too but I couldn't see a great way to fix it that late at night. I'll see what I can do with it, but suggestions are welcome. – Glen_b Mar 20 '13 at 01:41
  • @rolando2 In the end I just dropped the footnote. Better suggestions are welcome. – Glen_b Mar 20 '13 at 01:59
  • @Glen_b - I think this is a good revision. Cheers ~ – rolando2 Mar 22 '13 at 19:56
0

With such a large sample, very small changes will show up as sig. That does not necessarily make them clinically important; you have to interpret this in the light of what your dependent variable is. You said you're using Partial Eta Squared as your effect size measure. SPSS gives it as the option for effect sizes, so lots of people click on it, but it seems to create a lot of confusion. Cohen's d is a standardised alternative that will enable you to compare across different interventions and variables because its a standardised measure (varies between 0-3). Cohen's d is simple to calculate (its mean1-mean2/SD for related samples, or use the pooled variances formula for independent samples). Once you've got d, have a look at this brilliant effect size slider from @krstoffr: http://rpsychologist.com/d3/cohend/

jUST1N3
  • 91
  • 4