27

As the general consensus seems to be to use mixed-models via lmer() in R instead of classical ANOVA (for the often cited reasons, like unbalanced designs, crossed random effects etc.), I would like to give it a try with my data. However I am worried that I would be able to "sell" this approach to my supervisor (who is expecting classical analysis with a p-value in the end) or later to the reviewers.

Could you recommend some nice examples of published articles that used mixed-models or lmer() for different designs like repeated-measures or multiple within- and between-subject designs for the field biology, psychology, medicine?

chl
  • 50,972
  • 18
  • 205
  • 364
jokel
  • 2,403
  • 4
  • 32
  • 40
  • Also it would be nice to have some pointers to according references that are considered crucial to cite – jokel Apr 20 '12 at 21:56
  • 2
    On the [lme4 project](http://lme4.R-forge.R-project.org/) page, there is a [collection of articles and other research](http://lme4.R-forge.R-project.org/bib/lme4bib.html) using `nlme` or `lme4`. – chl Jul 04 '12 at 18:23

5 Answers5

26

Update 3 (May, 2013): Another really good paper on mixed models in Psychology was released in the Journal of Memory and Language (although I do not agree with the authors conclusions on how to obtain p-values, see package afex instead). It very nicely discusses on how to specify the random effects structure. Go read it!

Barr, D. J., Levy, R., Scheepers, C., & Tily, H. J. (2013). Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of Memory and Language, 68(3), 255–278. doi:10.1016/j.jml.2012.11.001

Update 2 (July, 2012): A paper advocating the use in (Social) Psychology when there are crossed (e.g., participants and items) random effects.
The big thing is: It shows how to obtain p-values using the pbkrtest package:

Judd, C. M., Westfall, J., & Kenny, D. A. (2012). Treating stimuli as a random factor in social psychology: A new and comprehensive solution to a pervasive but largely ignored problem. Journal of Personality and Social Psychology, 103(1), 54–69. doi:10.1037/a0028347
(only available as a Word .doc)

Jake Westfall told me (per mail) that an alternative for obtaining p-values to the advocated Kenward-Rogers approximation (used in pbkrtest) is the (less optimal) Satterthwaite approximation, which can be found in the MixMod package using the anovaTab function.

Small update to last update: My R package afex contains function mixed() to conveniently obtain p-values for all effects in a mixed model. Alternatively, the car package now also obtains p-values for mixed models in Anova() using test.statistic = "F"

UPDATE1: Another paper describing lme4

Kliegl, R., Wei, P., Dambacher, M., Yan, M., & Zhou, X. (2011). Experimental effects and individual differences in linear mixed models: estimating the relationship between spatial, object, and attraction effects in visual attention. Frontiers in Quantitative Psychology and Measurement, 1, 238. doi:10.3389/fpsyg.2010.00238

Original Response:

I do not have a number of examples, only one (see below), but know some paper you should cite from Psychology/Cognitive Sciences. The most important one is definitely:

Baayen, R. H., Davidson, D. J., & Bates, D. M. (2008). Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language, 59(4), 390–412. doi:10.1016/j.jml.2007.12.005

Another one from Baayen is:

Baayen, R. H., & Milin, P. (2010). Analyzing Reaction Times. International Journal of Psychological Research, 3(2), 12–28.

I actually totally liked his book, too, which also has a nice introductory chapter on mixed model (and is pretty cheap for a stats book):
Baayen, R. H. (2008). Analyzing linguistic data : a practical introduction to statistics using R. Cambridge, UK; New York: Cambridge University Press.

I probably guess he also has a lot of papers using lme4, but as my main interest is not psycholinguistics, you might wanna check his homepage.

From my field (reasoning), I know of this one paper that uses lme4:

Fugard, A. J. B., Pfeifer, N., Mayerhofer, B., & Kleiter, G. D. (2011). How people interpret conditionals: Shifts toward the conditional event. Journal of Experimental Psychology: Learning, Memory, and Cognition, 37(3), 635–648. doi:10.1037/a0022329

(although I have the feeling they use a likelihood ratio test to compare models which only differ in the fixed parameters, which I have heard is not the correct way. I think you should use AIC instead.)

Henrik
  • 13,314
  • 9
  • 63
  • 123
  • 3
    Draft of **Analyzing linguistic data : a practical introduction to statistics using R** is available [here](http://www.ualberta.ca/~baayen/publications/baayenCUPstats.pdf). – MYaseen208 Apr 21 '12 at 17:15
  • I also had the pdf, but as the book was really cheap I bought it and am really happy. It is kind of easier to read if you have it as a book. – Henrik Apr 21 '12 at 18:41
  • @Henrik you say LRTs should not be used to compare models that differ only in fixed effects, do you have a reference for this? – Matt May 06 '12 at 04:58
  • @Matt I have this information from a discussion with Mike Lawrence from the `ez` package: https://groups.google.com/forum/#!topic/ez4r/GQTEtNziRwE Mike cites Pinheiro & Bates (2000) for it, see the link. – Henrik May 07 '12 at 11:23
12

This is a highly-cited paper on mixed models for ecology and evolution:

Freya Harrison
  • 3,212
  • 4
  • 25
  • 31
7

The following article endeavours to promote the use of multilevel modelling in social science settings:

  • Bliese, P. D. & Ployhart, R. E. (2002). Growth Modeling Using Random Coefficient Models: Model Building, Testing, and Illustrations, Organizational Research Methods, Vol. 5 No. 4, October 2002 362-387. PDF

To quote the abstract:

In this article, the authors illustrate how random coefficient modeling can be used to develop growth models for the analysis of longitudinal data. In contrast to previous discussions of random coefficient models, this article provides step-by-step guidance using a model comparison framework. By approaching the modeling this way, the authors are able to build off a regression foundation and progressively estimate and evaluate more complex models. In the model comparison framework, the article illustrates the value of using likelihood tests to contrast alternative models (rather than the typical reliance on tests of significance involving individual parameters), and it provides code in the open-source language R to allow readers to replicate the results. The article concludes with practical guidelines for estimating growth models.

An examination of the articles listed on Google Scholar as citing this paper suggest several other useful leads.

Jeromy Anglim
  • 42,044
  • 23
  • 146
  • 250
  • This paper looks really interesting. Unfortunately it only uses `lme` from `nlme` instead of `lmer` from `lme4`. (+1) – Henrik Apr 23 '12 at 14:04
4

I am reading Zuur, A. F., Ieno, E. N., Walker, N., Saveliev, A. A., & Smith, G. M. (2009). Mixed effects models and extensions in ecology with R. New York, NY: Springer Science+Business Media, LLC. It is written for ecologists, so the stats are fairly easy to follow; I think it would be useful for people from other disciplines, such as medicine or psychology too. There are many case studies included, and each has a detailed section on how to best write up the stats in a paper.

gung - Reinstate Monica
  • 132,789
  • 81
  • 357
  • 650
Frieda
  • 53
  • 1
  • 5
3

An excellent example of using mixed models in ecology is:

Unfortunately it uses older R libraries.

N Brouwer
  • 1,786
  • 2
  • 18
  • 32