7

I am trying to predict y with variables a, b and c. I have two models and I get different results depending on how I fit my model.

Model A is the simpler model, in which I exclude variable c. In model B, I include all main effects and possible interactions between the 3 variables. I am interested in the ab interaction that is bolded.

In model A, there is NO significant ab interaction.

  • Model A: y ~ a + b + ab

In model B, there is a significant ab interaction, and the abc interaction is significant as well.

  • Model B: y ~ a + b + c + ab + ac + bc + abc

Everything has been coded properly, such that the variables are mean centered or effect coded. This is a generalized (logistic) linear mixed effects model fit using glmer in R.

My question is, is there a significant ab interaction? If I trust model A, then I would say no, but if I trust model B, then I would say yes. Which model is more correct, or does it depend on details of the model? Why does adding a 3-way interaction make an originally non-significant two-way interaction (model A) significant (model B)?

Alexandra Decker
  • 71
  • 1
  • Welcome Alexandra. These are two different models of reality (in a sense they are the same model, but in model A the coefficients on all the terms containing c are constrained to be zero). Determining which one, if either, is correct relies on substantive knowledge of the domain you are studying (see Fine Point 6.3 about (algorithmic) "causal discovery" in Hernán & Robins, 2019 [*Causal Inference*](https://www.hsph.harvard.edu/miguel-hernan/causal-inference-book/) for some decent points on this. – Alexis Jan 23 '19 at 02:24
  • Thanks for your help, @Alexis and for the reference. So, it seems like I will need to decide which is the most appropriate model, based on my domain knowledge. – Alexandra Decker Jan 23 '19 at 02:42
  • Sez me. :) And some other folks. – Alexis Jan 23 '19 at 02:54

1 Answers1

5

My question is, is there a significant ab interaction? If I trust model A, then I would say no, but if I trust model B, then I would say yes. Which model is more correct, or does it depend on details of the model? Why does adding a 3-way interaction make an originally non-significant two-way interaction (model A) significant (model B)?

Neither model is "correct". They are different models. It is absolutely OK to find a meaningful interaction in the first model, but the same interaction may not be meaningful in the second. In the second model, the a:b interaction itself is involved in an interaction with c. This means that the intepretation of the a:b interaction in the second model is conditional on c being zero (or at it's reference level if it is categorical). It is then necessary to intepret the a:b:c interation, which will tell you how the a:b interaction changes with c. So from this it is easy to see there is no reason to expect the a:b to have a similar magnitude (or statistial significance) in the two models.

Robert Long
  • 53,316
  • 10
  • 84
  • 148