0

We're using MuMIn in R to look at the delta R-squared when adding a term into a mixed model like this:

> mod1 <- lmer( DV ~ IV1+IV2+IV3+IV4+IV5+IV6+IV7+IV8+IV9 + (1|IV9:ID), data=dF, REML=FALSE)
> mod2 <- lmer( DV ~ IV1+IV2+IV3+IV4+IV5+IV6+IV7+IV8+IV9 + (1|IV9:ID) + IV10, data=dF, REML=FALSE)
> r.squaredGLMM(mod1)
           R2m       R2c
[1,] 0.1878124 0.6135028
> r.squaredGLMM(mod2)
          R2m       R2c
[1,] 0.320143 0.6137161

So from looking at the marginal pseudo-r-squared (R2m), which to my understanding looks at variance explained by fixed factors only, the addition of the single fixed-factor IV10 explains an appreciable chunk of extra variance. That makes sense. What I don't understand is how in the second model, the conditional pseudo-r-squared (R2c), which to my understanding looks at all the variance explained, increases hardly at all, despite the increase in marginal variance explained. The only way I understand this could be possible is if the variance explained by random factors went down by roughly the same amount as variance explained by fixed factors went up. But the random component is the same in each model. What gives?

EDIT: In response to Frans' question, I tried the models again without nesting ID in IV9 (i.e. without bothering to nest participant ID in country), so (1|ID) instead of (1|IV9:ID) and the results are completely the same. I think we were nesting ID inside country in case participants from different countries had the same ID code, but this result suggests they don't. Anyway, it's apparently not about the nesting issue.

EDIT2: In response to a question, here is the variance

> VarCorr(mod1)
 Groups     Name        Std.Dev.
 Country:ID (Intercept) 0.86373 
 Residual               0.82301 
> VarCorr(mod2)
 Groups     Name        Std.Dev.
 Country:ID (Intercept) 0.71748 
 Residual               0.82301

EDIT 3: It's exactly the same in the version with no nesting. I think nesting is unnecessary for this model, it was done just in case ID numbers were duplicated across countries, so the nesting issue is a red herring:

> VarCorr(mod1)
 Groups   Name        Std.Dev.
 ID       (Intercept) 0.86373 
 Residual             0.82301 
> VarCorr(mod2)
 Groups   Name        Std.Dev.
 ID       (Intercept) 0.71748 
 Residual             0.82301
Amorphia
  • 591
  • 1
  • 3
  • 13
  • 1
    What is your random effect? `IV9:ID` is an unfamiliar notation to me, did you intend for a random slope (`IV9 | ID`), or to tell R which variables are nested in the random effect (you don't need to specify this), or something else? Also, what is `IV10`? Does it perhaps share information with the random effect? – Frans Rodenburg Jan 22 '20 at 10:29
  • Somewhat confusingly because colon is also used for interactions, you can use it for nesting in this context (see e.g. https://stats.stackexchange.com/questions/232109/does-it-make-sense-for-a-fixed-effect-to-be-nested-within-a-random-one-or-how-t?noredirect=1&lq=1). IV9 is what country a participant is in, and ID is their participant ID. Participant ID is the only random factor, but it's nested in IV9, which is their country. – Amorphia Jan 22 '20 at 11:07
  • 1
    How about `IV10`, is it perhaps similar to the information contained in the random effect? That would explain the discrepancy. If it doesn't, perhaps your example is just a demonstration of the limitations of pseudo-$R^2$. – Frans Rodenburg Jan 22 '20 at 11:16
  • 1
    What is the reasoning behind having IV9 in both the fixed part of the model and in the random part of the model? Although that is allowed, you typically see a variable in one or the other, unless you are allowing the slope of a predictor to vary across clusters, which would look like `(IV9 | ID)`. – Erik Ruzek Jan 22 '20 at 14:26
  • @FransRodenburg no way, IV10 is a continuous variable where each participant (the random effect ID) has quite different DV values across trials. But I think this was a good idea if you didn't know that. I'm starting to agree that this is a limitation of pseudo-R-squared (or maybe this implementation?). – Amorphia Jan 22 '20 at 15:01
  • @ErikRuzek thanks, this is explained in the edit now. The notation I have used just means ID is nested in IV9. But it doesn't matter if I remove this nesting, the results are the same. – Amorphia Jan 22 '20 at 15:02
  • 1
    Can you post the random effect variance estimates from these two models? It is strange that the results are the same. – Erik Ruzek Jan 22 '20 at 21:27
  • @ErikRuzek thanks, updated above. Do you think this could be a bug in MuMIn? Should I report this do you think? – Amorphia Jan 23 '20 at 08:09
  • Sorry, I also wanted to see the random effect variance estimate from the `(1|ID)` model. – Erik Ruzek Jan 23 '20 at 14:49
  • Thanks @ErikRuzek - it's exactly the same, now posted above. – Amorphia Jan 23 '20 at 18:47
  • Actually, it's quite far from the same: `mod1` explains considerably more variance through the random intercept than `mod2`, which may indicate that indeed, the addition of `IV10` just explains more variance in the fixed part by accounting for variance that was otherwise accounted for in the random part. – Frans Rodenburg Jan 26 '20 at 14:07

0 Answers0