3

I've been working on a messy, repeated measures data set of endocrine data looking at a small group of variables (after eliminating several uninteresting contenders in exploratory data analysis), each of which is linked to a specific hypothesis about factors that lead to hormone expression. Information-theoretic reasons have pushed me to the full enchilada model (including the optional guac), which leaves me with a bit of conundrum.

Obviously, the issue of p-values in mixed effects models is one that's been well hashed, and I'm willing to accept much of the logic that's been put forward. df isn't naively calculable? Awesome, count me onboard. But in my my last model standing, how do I interpret whether my betas support or refute the predictions of these hypotheses? In a non-mixed effects context, I'd be looking to a p-value from my betas + SE to see if the predictions of the hypothesis(es) were supported, or whether my betas aren't different from 'no effect'.

Or, am I thinking about this all wrong? Because my top models (via AICc) include all the terms, do I therefore conclude that all three variables are biologically meaningful and therefore am only concerned with the direction of the betas?

Glen_b
  • 257,508
  • 32
  • 553
  • 939
twoyaks
  • 51
  • 3

1 Answers1

2

As someone else in the endocrine field, I think your answer will depend on where you want to publish. Most high level journals, especially if they tend towards a clinical audience, will want p-values. While I am a big fan of R for providing many opportunities to apply more critical evaluation of "usual statistics," abandoning p-values is one of the things that a reviewer or committee member may simply never agree to.

That being said, here are a couple work arounds that I think will be more palatable than AIC and avoid coefficient-specific p-values.

  1. Compare the simpler models to continually more complex models using the likelihood ratio test. My understanding is that this does not suffer from the same problem as determining coefficient p-values. Perhaps the most complex model will be best - you could then use standardised coefficients to help interpret relative importance of each variable.

  2. Construct confidence intervals for each coefficient (using normal theory or turn to bootstrap if you have a small sample size/ doubt that your data approximate normality).

You can find these answers regarding p-values in the lme4 package here. It also discusses ways of obtaining p-values that might convince you of using them after all.

Moose
  • 1,090
  • 7
  • 12