Do I need to make an alpha adjustment when testing planned orthogonal contrasts. While the contrasts are independent the same residual error is used. The author of a well regarded textbook on experiment design in behavioral research, Roger Kirk, states that "If orthogonal contrasts have been planned in advance, contemporary practice favors adopting the contrast as the conceptual unit for a Type I error." He also cited, “Research by Norton and Bulgren, as cited by Games (1971), indicates that when the degrees of freedom for MSerror are moderately large, say 40, multiple t tests can, for all practical purposes, be regarded as independent.” (section 5.2). Does anyone know if this is regarded as "contemporary practice" in clinical trials research where family wise error rate is important to control?
Asked
Active
Viewed 51 times
1
-
When not corrected, Type I error inflation is more of a problem with independent tests than with dependent tests. Your sources are wrong. – BigBendRegion Jan 08 '21 at 12:20
-
Would this apply to planned comparisons in a one-way ANOVA? What about main effects in a factorial ANOVA when factors are orthogonal? – TTemplln Jan 15 '21 at 05:56
-
The answer is yes: if uncorrected, the familywise type I error rate is always higher for independent tests than for dependent tests. Further, the higher the correlations, the less the error rate. Thus, multiple comparisons methods are needed less with correlated tests than with independent tests. It does not matter what kind of experimental design or analysis model is used. – BigBendRegion Jan 15 '21 at 12:54