I conducted one-way anova test and get a significant result, but when I conducted Scheffe test there was no significant result. Is this possible? How should I proceed or report the results?
Asked
Active
Viewed 1,216 times
2
-
Yes, it's possible. There are many questions here on related issues. I'll see if I can dig one or two up. – Glen_b Nov 20 '13 at 22:51
-
1@Glen_b One of them might be http://stats.stackexchange.com/questions/14500. – whuber Nov 20 '13 at 23:01
-
2@whuber Thanks, yes, that's relevant. There's also [this](http://stats.stackexchange.com/questions/29248/strange-result-of-post-hoc-test), and it can also happen [the other way around](http://stats.stackexchange.com/questions/35175/non-significant-group-effect-but-lsd-post-hoc-is-significant-why) (though you wouldn't normally do a post hoc in that case). – Glen_b Nov 20 '13 at 23:04
1 Answers
2
Scheffe's test allows you to do more than compare group means. You can compare the average of groups A and B with the average of groups C, D and E. Etc. The only real reason to choose Scheffe's test for multiple comparisons is to do that kind of contrast. If you only compare group means, this test has less power than others (Tukey, Dunnett).
The Scheffe's test (unlike Tukey etc.) guarantees you that if the overall ANOVA is statistically significant then at least one contrast will be too (page 217, Maxwell and Delaney, 2nd edition). But that contrast may not be a simple comparison of group means, but rather may be a complex comparison involving many groups.
Harvey Motulsky
- 14,903
- 11
- 51
- 98