0

I have two factors, and I've fitted a interaction model in R with $lm( \sim factor1*factor2)$. The parameters belonging to the interactions between the two factors are all non-significant (p-values between 30% and 50%). There are two such parameters.

When I then run a F-test between this model and the additive model ($lm(\sim factor1 + factor2)$), I get a p-value of about $10 \%$.

Question: Why is this $p$-value so low? It seemed from the first couple of $p$-values that I could comfortably accept my hypothesis, but now, I am not so sure? Should I still accept the hypothesis and try to work in the additive model, or is the interaction still significant?

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
Getri
  • 1
  • 1
    This is a version of the question addressed at http://stats.stackexchange.com/questions/14500. It illustrates why we use F tests rather than looking at p-values of individual coefficients when coefficients have to be considered as groups. – whuber May 11 '16 at 20:17

1 Answers1

1

First, consider the simpler case of your model but with continuous variables. Say, something like:

    mod <- lm(y ~ x1+x2)

Your regression coefficient for $x_1$ is just the partial derivative of $y$ with respect to $x_1$. This is the marginal effect, but because everything else is a constant, you just end up with $b_1$.

However, when we specify an interaction like you have, what we're really specifying is this:

    mod2 <- lm(y ~ x1+x2+x1*x2)

When you have a interaction term, you also have to have the additive terms. This is why you get so many regression coefficients in your summary. So for this model with continuous variables, summary() would give us an intercept, a coefficient for $x_1$, a coefficient for $x_2$, and a coefficient for $x_1*x_2$. But this time, when we take the partial derivative of $y$ with respect to $x_1$, we don't just get a single coefficient. Instead, the marginal effect is $b_1 + b_3*x_2$. So the effect of $x_1$ depends on the value of $x_2$ at any point. As a result, the p-values in your summary() don't really tell you very much. It's a common mistake to think that you can interpret your coefficients the same way in an interaction model. The coefficient $b_3$ does not explain the 'variance due to the interaction'. Brambor, Clark and Golder do a great job of explaining just how common this mistake is, and its perils.

Now, your question had to do with factor variables. The explanation is the same; it's just that you have many more coefficients, because you're multiplying many binary variables with many other binary variables. Be cautious here - you'll see your $R^2$ go up, but that may just be due to having many more terms in your model.

A common approach to reporting on interaction effects is to plot predicted values for various values of your interacted variables, holding other values at their mean (or some other sensible values). There's a nice package in R that will do this for you. Try:

    library(effects)
    plot(allEffects(mod))

As for which model you should go with - I'd say that should depend on your theory more so than anything else. Does an interaction make sense for what you're trying to model? I wouldn't suggest adding interaction terms just to get better fit statistics$\dots$

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
5ayat
  • 527
  • 3
  • 15
  • 1
    This is a good answer. I wouldn't call the regression coefficient for x1 the "marginal effect", though. It is still conditional on the other variables in the model. It isn't necessarily the same as the coefficient you would get if you only had x1 in the model, eg. – gung - Reinstate Monica May 11 '16 at 21:36
  • 1
    Thanks. It is still a marginal effect. For the simple linear y = a + b1x1+b2x2 + e, dy/dx1 = b1. It's just such a trivial operation that we often forget to tell students that they're actually taking a derivative when they're reporting a regression coefficient. It's the effect of x1 with x2 held constant. So for the OP, interactions are just a small extension of what they already know - it's just that the derivative is *slightly* more complicated, but the fundamentals remain the same. – 5ayat May 12 '16 at 07:22
  • Your link only leads to journal frontpage, not actual paper ... – kjetil b halvorsen Apr 17 '21 at 16:01