3

I have run a sequence of nested multi-level models using lmer, and looked at both the summary output giving all the coefficients, etc, as well as the Anova output which does a chi-squared test on each of the factors.

In the first model without the interaction, the factor medium is significant, looking at both the chi-squared test output and at the coefficients from the summary output.

In the second model when the interaction is added, the first-level factor medium is no longer significant in the chi-squared test output yielded by the Anova function. (It isn't significant in the summary output where we get the coefficient information, but I believe I understand how to interpret that output - it just means that the medium is not significant for the reference group I believe?)

But how do I interpret the non-significant outcome of the chi-squared test for the first-level factor medium in the presence of the added interaction?

If it helps, here is a simplified version of my model:

success~medium+gender+gender*medium

And here is the Anova (chi-squared tests) output for Model 1 (without the interaction) and Model 2 (with the interaction):

Model 1:

                Chisq           Df  Pr(>Chisq)
medium          42.03818323     1   0.0000
gender          6.082823479     1   0.0137

Model 2:

                Chisq           Df  Pr(>Chisq)
medium          2.23141795      1   0.1352
gender          12.95254908     1   0.0003
medium:gender   6.920056392     1   0.0085

I'd be happy to read a reference on this, but I just can't find anything in any of my books on how to interpret the chi-squared tests in the presence of an interaction - they all seem to focus entirely on interpreting the regression coefficients only. So if anyone could help to explain the correct interpretation, or refer me to a good reference that specifically talks about how to interpret the chi-squared tests of main effects in the presence of interactions, I'd be grateful!

cww
  • 151
  • 1
  • 8
  • Is your issue that you understand the lack of significance when you call `summary(model2)`, but do not understand the lack of significance when you call `Anova(model2)`? Are the p-values for `medium` different for these two outputs? (In general, the interpretation should be the same.) – gung - Reinstate Monica Sep 22 '13 at 16:54
  • Thanks @gung, yes that is exactly where I am unclear. I thought that the meaning of the coefficients in summary(model2) was the average "effect" of that factor on the reference group, and that for the chi squared tests in the Anova(model2) output, this was a test of the significance of the improvement in model fit when that factor was added to the model. But because we can't exclude the factor “medium” from the model without excluding medium:gender from the model also, I don’t understand which two models the chi-squared test is comparing? – cww Sep 22 '13 at 18:41
  • To answer your question about p-values: In the simple model given above with just medium and gender, the p-value for the summary and Anova outputs for medium are in fact the same, but in a more complicated version of the model I have gotten very different values from Anova and summary outputs for the same factor (e.g. 0.04 vs. 2.2E-11), which reinforced my impression that these were measuring something different? – cww Sep 22 '13 at 18:43
  • If there is no interaction, the coefficient in the `summary` output is the amount you shift the regression line up or down (w/o changing the slope); w/ interaction, it's the shift *only* when the other var = 0. Re "can't exclude the factor" in `Anova`, you *shouldn't*, but it is possible (see [here](http://stats.stackexchange.com/questions/11009/)). Re outputs sometimes same / sometimes different, that depends on the 'type' of SS you are using (see [here](http://stats.stackexchange.com/questions/20452//20455#20455)) p's should be the same when using type III SS or when the vars are orthogonal. – gung - Reinstate Monica Sep 22 '13 at 19:03
  • Thanks @gung for the links! I understand type III SS better now, but I am still unsure about what I can legitimately say about the main effect in the presence of the interaction. I want to say something like: The effect of medium is largely accounted for by the effect of medium on particular subgroups for which the interaction with medium is significant (here, differences in outcomes by medium can be largely accounted for by differences in outcomes for women by medium). But I’m not sure if a statement like this is right (especially when there are multiple interactions or non-binary factors)? – cww Sep 22 '13 at 21:00
  • I'm not sure I follow your statement. In general, when there's an interaction, we *don't* interpret the main effects, though. Instead, you interpret the *simple effects*. Ie, you stratify your model on 1 of the terms (whichever makes more sense for you) and explain / interpret the relationship b/t the other term & the response at each level of the stratified term. If your covariates are continuous, you can specify the relationship b/t x1 & y @ the mean of x2 & +/- 1SD. For general info on interactions my answer [here](http://stats.stackexchange.com/questions/49002//49701#49701) might help. – gung - Reinstate Monica Sep 22 '13 at 21:39

0 Answers0