1

I have a bit of a theoretical question about random effects models and regression. If I have a set of clustered, longitudinal data (say repeated measurements of $y$ on a number of different individuals) is there really any difference, aside for adjustments to degrees of freedom, perhaps, to fitting a regression model for each individual and averaging the parameter estimates across models to get an overall average model (i.e. all fixed effects) vs. fitting a model with individual id as a random effect in which each of the individual's observations fall (i.e a mixed model)?

Thanks.

StatCurious
  • 157
  • 9
  • Are you assuming you have an identical number of points per individual, or that you are doing a weighted averaging afterwards? – mkt Jul 21 '19 at 19:53
  • Incidentally, your question could also be reframed as: "What's the difference between including individual as a fixed effect vs. a random effect?". And I discuss this a bit in my answer here: https://stats.stackexchange.com/a/289346/121522 – mkt Jul 21 '19 at 19:55
  • @hi, mkt, I should have been more clear that I'm assuming equal sizes or weighted averaging. Thanks for the link. I'll give it a read now. – StatCurious Jul 21 '19 at 20:42
  • See Gelman's [the Secret Weapon](https://statmodeling.stat.columbia.edu/2005/03/07/the_secret_weap/). – kjetil b halvorsen Nov 26 '19 at 02:41

1 Answers1

2

Here's 3 cases to consider:

  1. Separate linear regressions per individual
  2. A single regression with a fixed effect for individual and an interaction term allowing individuals to have different slopes
  3. A mixed model with random individual-level intercepts and slopes

1 and 2 are equivalent in terms of their mean predictions, though you're correct that the degrees of freedom & standard errors will differ. Since you don't care about that, I'll treat them as identical for purposes of comparing them against 3.

3 is different because of how random effects work: the individual-level random effects are shrunk towards zero i.e. you will tend to get less extreme values than you did in 1 & 2. This shrinkage is an important reason why random effects are valuable; it's not just about saving degrees of freedom.

A good description of why this works is in this old article on Stein's paradox in statistics

mkt
  • 11,770
  • 9
  • 51
  • 125