34

The lmerTest package provides an anova() function for linear mixed models with optionally Satterthwaite's (default) or Kenward-Roger's approximation of the degrees of freedom (df). What is the difference between these two approaches? When to choose which?

amoeba
  • 93,463
  • 28
  • 275
  • 317
doko
  • 441
  • 1
  • 4
  • 4
  • 4
    See the companion paper [Kuznetsova et al, 2017, lmerTest Package: Tests in Linear Mixed Effects Models](https://www.jstatsoft.org/article/view/v082i13/v82i13). – amoeba Jan 07 '18 at 15:02
  • 2
    In the Discussion they say "From our practice, we observed that the p values that the approximation methods provide are generally very close to each other. Schaalje, McBride, and Fellingham (2002) performed a number of simulations in order to investigate the appropriateness of the approximation methods. They discovered that complexity of the covariance structures, sample size and imbalance affect the performance of both approximations. However, these factors affect the Satterthwaite’s method more than the Kenward-Roger’s." – amoeba Jan 07 '18 at 15:20
  • Two examples where KR gives more appropriate dfs than Satterthwaite: https://stats.stackexchange.com/questions/320895 and https://stats.stackexchange.com/questions/84268. – amoeba Jan 13 '18 at 17:45
  • Another example: https://stats.stackexchange.com/questions/331541. – amoeba Mar 04 '18 at 17:47
  • 1
    The article [Evaluating significance in linear mixed-effects models in R by Steven G. Luke](https://link.springer.com/article/10.3758%2Fs13428-016-0809-y) has some nice comparisons of these methods. It concludes that both KR and Satterthwaite derived from REML models produce acceptable Type I error rates even for smaller samples. – cbrnr Jun 14 '18 at 09:06

2 Answers2

8

I'm also interested in figuring out what the difference might be. The best I can offer you, for now, is that this blog post suggests that the Kenward-Roger approximation is slightly, but probably not significantly, more conservative than the Satterthwaite approximation. The author also notes that they are both more conservative than the normal approximation, but again, not by much if the sample size is high enough. I'm not sure whether or not this was a generalizable conclusion of the author's or not though.

Edit: I will add that the article "A comparison of denominator degrees of freedom approximation methods in the unbalanced two-way factorial mixed model" by K.B. Gregory seems to indicate that neither method is typically a better method, although there are apparently occasions where the Kenward-Roger approximation loses some level of conservativeness.

Jan Kukacka
  • 10,121
  • 1
  • 36
  • 62
Bajcz
  • 515
  • 1
  • 4
  • 19
  • 3
    it's *Kenward-Roger* (no "s") ... Kenward-Roger's if you insist ... but usually expressed without the 's ... see also http://link.springer.com/article/10.1198/108571102726 – Ben Bolker Jan 04 '15 at 23:00
6

Another difference between the two methods is described in Luke (2017):

Both the Kenward-Roger (Kenward & Roger, 1997) and Satterthwaite (1941) approaches are used to estimate denominator degrees of freedom for F statistics or degrees of freedom for t statistics. SAS PROC MIXED uses the Satterthwaite approximation (SAS Institute, 2008). While the Satterthwaite approximation can be applied to ML or REML models, the Kenward-Roger approximation is applied to REML models only.
(my bold)

gung - Reinstate Monica
  • 132,789
  • 81
  • 357
  • 650
Angel Lu
  • 101
  • 1
  • 3