This is a follow-up question to this one.
My dataset consists of:
one dependent variable (
value) measured 3 times (observation) by 3 people (evaluator) on the 2 sides (side- right/left) of anindividual(~150 individuals)several independent variables: one continuous (
age), and 2 discrete (sex,ancestry).
Based on advice by @RobertLong from the previous question, I fitted a mixed model using lme4, which works really well to see the effect of the different independent variables:
m0 <- lmer(value ~ sex * side + age + ancestry + evaluator
+ (1 | individual), data = L, REML = F, na.action = na.exclude)
Now, I would like to calculate ICC (2,k: 2-way random-effects model with mean of k-raters) for inter and intra reliability measurements, so evaluator and observation respectively. Obviously, the mixed model above only has individual as a random effect (bc observation has no repeated measurements and evaluator only has 3 levels).
Could I create a model with just random effects and use the variances to calculate the ICC?
Where ICC = (variance of interest) / (total variance)
m1<- lmer(value ~ 1 + (1|evaluator) +(1|observation) + (1|individual)
+ (1|individual:evaluator) + (1|individual:observation) + (1|observation:evaluator),
data = L, REML = F, na.action = na.exclude)
Random effects:
Groups Name Variance Std.Dev.
individual:evaluator (Intercept) 11.97325 3.4602
individual:observation (Intercept) 0.00000 0.0000
individual (Intercept) 39.67049 6.2985
observation:evaluator (Intercept) 3.35913 1.8328
evaluator (Intercept) 21.22271 4.6068
observation (Intercept) 0.01308 0.1144
Residual 48.65471 6.9753
Number of obs: 3180
This model is singular btw. From here, I'm thinking this is not a good idea, but would like to understand better why and of any alternatives.
