Understanding Kenward-Rogers Degrees of Freedom from lsmeans()

Question

I'm trying to analyze data using a multilevel model which predicts subject response times using the experimental group and trial validity.

library(lmerTest)
rt.model <- lmer(RT ~ Group+Valid +(1 +Valid||subject), data=rt_data)

The model resulted in a significant interaction between group and validity, so to probe the interaction I followed the suggestion here to create an interaction model like this:

rt_data$interaction<-interaction(rt_data$Valid, rt_data$Group)
rt.int.model <- lmer(RT ~ interaction +(1 +Valid|PAR), data=rt_data)

and did multiple comparisons using lsmeans()

lsmeans(rt.int.model, pairwise~interaction)

and I get the following output

$lsmeans
interaction    lsmean         SE    df  lower.CL upper.CL
0.Color     1.3657214 0.06427078 51.98 1.2367516 1.494691
1.Color     0.9742036 0.06427078 51.98 0.8452339 1.103173
0.Motion    1.6172733 0.06097262 51.98 1.4949218 1.739625
1.Motion    1.4595154 0.06097262 51.98 1.3371639 1.581867

Degrees-of-freedom method: kenward-roger 
Confidence level used: 0.95 

$contrasts
contrast               estimate         SE    df t.ratio p.value
0.Color - 1.Color    0.39151778 0.05599065 36.00   6.993  <.0001
0.Color - 0.Motion  -0.25155189 0.08859116 51.98  -2.839  0.0317
0.Color - 1.Motion  -0.09379399 0.08859116 51.98  -1.059  0.7158
1.Color - 0.Motion  -0.64306967 0.08859116 51.98  -7.259  <.0001
1.Color - 1.Motion  -0.48531177 0.08859116 51.98  -5.478  <.0001
0.Motion - 1.Motion  0.15775790 0.05311739 36.00   2.970  0.0259

P value adjustment: tukey method for comparing a family of 4 estimates

In my data set, I have 18 people in the color group and 20 people in the motion group.

So my question is this: why do I get DF = 36 for within group comparisons for this method?

Also, is there any benefit to using this method over paired t-tests between subgroups and then correcting the p-value?

Thanks in advance!

Russ Lenth · Answer 1 · 2018-08-28T00:42:12.997

Question 1. Why shouldn’t it be? The fact that there are random grouping effects usually means that some comparisons have different degrees of freedom than others. It is more common that within-group comparisons would have more, not fewer, df. But that’s what happens when there are random intercepts; and this model is more complex than that.

Question 2. You cannot expect the same results from pairwise t tests, because those tests represent a different model. For one thing, your present model assumes homogeneity of certain variances, which in turn are estimated using all of the data. With the pairwise t tests, each test is based on only a subset of the data, different subsets for each test. So these are whole different animals.

Understanding Kenward-Rogers Degrees of Freedom from lsmeans()

1 Answers1