2

This is the current data I have

enter image description here

I want to know if fixed factor A has any effect on C. That is, if A1 = A2 = ... = A5.

I think I am trying to find a group difference, correct me if I am wrong.

because I read a paper saying it is fine to treat Likert scale in parametric, I changed factor C into numeric., and I decided to use mixed-effects model through

library(lme4)
library(lmerTest)
model <- lmer(C ~ A + (1|B), data) 
summary(model)
anova(model)

enter image description here

and reached the conclusion: we cannot reject the idea that factor A is consistent in manner of variability in resultC.

Then I read about the controversies behind p-values in mixed model, then eventually question whether that p-value actually represents my question, or if it answers a completely different question.

If I want to answer my question of A1 = A2 = ... = A5, is my use of a mixed model correct?

Thank you for your guidance.

aiorr
  • 137
  • 4

1 Answers1

1

I think your approach for examining whether the levels of A have an equivalent or near-equivalent effect on C make sense. In addition to your ANOVA approach, you could run a second model that excludes A as a predictor and then use a likelihood ratio test to see if the model without A is a better fit to the data:

model <- lmer(C ~ A + (1|B), data) 
model0 <- lmer(C ~ 1 + (1|B), data) 
anova(model0, model)

A significant likelihood ratio test would be an indication that the model that has A as a predictor provides a better fit to your data whereas an insignificant likelihood ratio test would indicate that A is not needed to model the outcome C. You also get AIC and BIC fit statistics for the two models, and these are useful for model comparison. Information criteria measures such as these are useful only in comparing two models. Lower values indicate better fit. See this thread for more information.

Erik Ruzek
  • 3,297
  • 10
  • 18