5

Suppose we have two variables $x_1$ and $x_2$ and an interaction term $x_1 \cdot x_2$. Suppose we set the family-wise error rate to $\alpha = 0.05$. For the Bonferroni correction, would we look at $\alpha/2$ or $\alpha/3$?

Nick Cox
  • 48,377
  • 8
  • 110
  • 156
user2886760
  • 51
  • 2
  • 1
    Just out of curiosity, are you also the person behind this question: [assessing-approximate-distribution-of-data-based-on-a-histogram](http://stats.stackexchange.com/q/51718/), & this question: [what-is-the-intuition-behind-conditional-gaussian-distributions](http://stats.stackexchange.com/q/71260/)? (The usernames seem similar to me.) If so, would you mind [registering](http://stats.stackexchange.com/help/creating-accounts) your account, & then [merging](http://stats.stackexchange.com/help/merging-accounts) these into 1 account? We're happy to help you, but this makes the site run smoother. – gung - Reinstate Monica Oct 17 '13 at 15:01
  • Why do you want to do that? In all of the scientific literature I know, different main and interaction effects of an ANOVA are *not* considered as members of the same family of comparisons. In other words, alpha is controlled for each effect by itself. Multiple comparisons are corrected for only when simple effects are tested. – Trisoloriansunscreen Oct 17 '13 at 19:47
  • 5
    That's true, @Tal, but it's simply due to tradition. That is, it's what people do because it's what people do. There is a legitimate question about whether it is the best way to go about things. Even if you believe the appropriate answer here is "no" (which is undoubtedly a defensible position), this is certainly a question worth asking. – gung - Reinstate Monica Oct 17 '13 at 20:11
  • You are doing three tests, so its $\alpha/3$. (I'd prefer the less conservative Bonferroni-Holm correction, which is almost as simple to apply). @Tal: Setting a global error in modelling would successfully avoid people doing p-value based variable selection... – Michael M Oct 18 '13 at 07:55
  • 1
    @Tal, this paper actually address the issue: Cramer, A. O. J., van Ravenzwaaij, D., Matzke, D., Steingroever, H., Wetzels, R., Grasman, R. P. P. P., … Wagenmakers, E.-J. (2016). Hidden multiplicity in exploratory multiway ANOVA: Prevalence and remedies. Psychonomic Bulletin & Review, 23(2), 640–647. https://doi.org/10.3758/s13423-015-0913-5 – Heteroskedastic Jim Jul 18 '17 at 14:51

1 Answers1

2

If you're testing the significance of $x_1$, $x_2$, and $x_1x_2$, then you're doing three comparisons, so if you want to use a Bonferroni (or Holm-Bonferroni) correction, it should correct for three comparisons, using $α/3$.

(As mentioned in the comments, you should use the Holm-Bonferroni rather than the Bonferroni, since the Holm-Bonferroni is uniformly more powerful and is applicable in all the same situations as the Bonferroni.)

Kodiologist
  • 19,063
  • 2
  • 36
  • 68