3

I have estimated a linear mixed model with REML and ML estimation. However, the estimated coefficients do not differ. The standard errors of the coefficients are slightly higher for the REML estimation. The regression output is the following: enter image description here

My question is why don't the estimated coefficients differ? I thought the ML estimation is unbiased in case of the fixed effects, but biased for the REMl estimation.

The estimated models in R are: REML: fit_mixed<-lmer(formula=Brutto ~ Alter + Geschlecht + AusVolPra+ Deutsch + Englisch + Schuljahre + Kind + Religion+ Kurs + Analphabet+(1|Herkunftsland), data = daten)

ML:fit_mixed2<-lmer(Brutto ~ Alter + Geschlecht + AusVolPra+ Deutsch+ Englisch + Schuljahre + Kind + Religion+ Kurs + Analphabet+(1|Herkunftsland), data=daten, REML=FALSE)

Herkunftsland is the random effect.

Furthermore I noticed that the variance of the random effect Herkunftsland is 0 for the REML estimation but non-zero for the ML estimation. Why is this the case?

Thanks in advance for your help.

Student
  • 33
  • 4

1 Answers1

3

Section 7.3 of the excellent book Bayesian Data Analysis in Ecology Using Linear Models with R, BUGS, and STAN by Fränzi Korner-Nievergelt et al. (2015) states the following in connection with REML and ML:

"For our purposes, the relevant difference between the two methods is that the ML estimates are unbiased for the fixed effects but biased for the random effects, whereas the REML estimates are biased for the fixed effects and unbiased for the random effects. However, when sample size is large compared to the number of model parameters, the differences between the ML and REML estimates become negligible. As a guideline, use REML if the interest is in the random effects (variance parameters) and ML if the interested is in the fixed effects."

Isabella Ghement
  • 18,164
  • 2
  • 22
  • 46
  • 2
    Thanks a lot that really cleared things up. Up until now I thought that the differences could only be negligible for the random effects. – Student Jul 01 '18 at 21:07
  • 1
    If you check out slides 8-9 of https://www.stat.wisc.edu/~ane/st572/notes/lec21.pdf, you'll find an example where the fixed intercept effect produced by REML and ML is identical, but its SE is smaller with ML. Similarly, the variance of the random Groups effect is smaller with ML. – Isabella Ghement Jul 01 '18 at 21:13
  • I meant random site effect... – Isabella Ghement Jul 01 '18 at 21:22
  • Thank you very much for the helpful material. I am still wondering why in the REML estimation the variance of the random effect Herkunftsland is 0. This is not the case in the slides you suggested. Thank you very much for your answers, I appreciate them. – Student Jul 01 '18 at 21:37
  • I did the ML estimation of the model and the variance of the random effect Herkunftsland was not zero. However, if I replicate the ML estimation, I get the result that the variance of the random effect is 0 too. How can that be, if the LM test clearly shows that the variance is not zero? – Student Jul 01 '18 at 21:39
  • How much bigger than 0 is the estimate you get for the ML estimation? – Isabella Ghement Jul 01 '18 at 22:05
  • 1
    I found some interesting answers for the 0 variance case here https://stats.stackexchange.com/questions/115090/why-do-i-get-zero-variance-of-a-random-effect-in-my-mixed-model-despite-some-va and here http://rpubs.com/bbolker/4187 – Student Jul 01 '18 at 22:46
  • Interesting indeed! Thanks for sharing the links! – Isabella Ghement Jul 02 '18 at 13:05