1

I have an experimental design for a GLMM as follows:

  • independent variable: fixed factor with 3 levels, randomly assigned between groups (condition a, condition b, condition c)
  • dependent variable: repeated measures over 4 trials with a dichotomous outcome (0,1) in each trial (trial 1, trial 2, trial 3, trial 4)
  • covariates: age (continuous), order (for counterbalancing; order 1, order 2)
  • participant ID (as random effects factor)

Similar studies (using t-tests or similar) have previously found an effect of around d = 0.6

I am struggling to understand how to calculate the sample size I need a priori. I have seen there are some packages available (e.g., simr, longpower, powerlmm, simglm), however, I think because I am in general a bit inexperienced with GLMMs I am having some difficulties in applying them to my example. I understand that I first need to create a simulated dataset, but I am not sure how to go about this.

I read through the following questions: Sample size calculation for mixed models

How can you compute sample size for a linear mixed model? G*Power only does repeated measures ANOVA

A priori power analysis for generalized linear mixed-effects model

Mixed effects model for power analysis to aid study design

I also tried following this tutorial but got stuck conceptually on how to create simulated data.

Could somebody point me in the right direction for how I could go about calculating power for this example?

becbot
  • 71
  • 5
  • Do you care about training / learning / fatigue effects (improvement / deterioration over trials)? Do you care about the covariates, or are they nuisance variables? Can you specify an effect size (ie, the probability of 'success')? – gung - Reinstate Monica May 03 '21 at 13:59
  • @gung-ReinstateMonica the effect size used in previous similar studies with t-tests is d = 0.6, I'm not interested in effects over trials (I was thinking it might also be possible to create a composite score out of 4, but I don't know if this is appropriate), for the covariates I am interested in the effect of age, but the counterbalancing is more of a nuisance variable to make sure we controlled for confounds. I hope that helps! – becbot May 03 '21 at 14:02
  • d=.6 doesn't make sense for binomials; the SD has to change if the probability of success changes, & the base rate is generally important for the power of tests of binomials. You need to stipulate the success rates you want to be able to differentiate. – gung - Reinstate Monica May 03 '21 at 14:07
  • I think previous studies created a score (from 1-4) out of the 4 trials, the dichotomous outcome is correct, incorrect, in which case I think there would be variation around the SD? – becbot May 03 '21 at 14:09
  • You need to stipulate the probability that a participant in a given condition (+ change associated w/ age, if you really care about the power for that) will get the outcome correct. If you have that, you can get a basic power calculation, if you don't, you can't. – gung - Reinstate Monica May 03 '21 at 14:12
  • I'm not sure I completely follow... previous similar experiments have used 2 conditions also with 4 trials, then used t-tests to compare the composite scores between the two groups. I am extending the design by adding a third condition, so in theory I guess I could also run an ANOVA, however, I thought a GLMM would be better to account for participants individual differences. I have hypotheses about which condition would have more correct responses, but I am unsure how I would get the specific probabilities in this case. – becbot May 03 '21 at 15:08
  • You need to stipulate the probabilities that you are interested in being able to differentiate. If you say there is a 1% chance people in A will be correct, & a 2% chance people in B will be correct, that will lead to some required N. If you say there is a 51% chance for A & a 52% chance for B, that will lead to a different required N. If you say there is a 40% chance for A & a 60% chance for B, that will lead to still another required N... You need to say what probabilities you want to be able to differentiate. That's how power analyses work. – gung - Reinstate Monica May 03 '21 at 15:14
  • Please state the probabilities of success you want to differentiate from each other. Do you understand how logistic regression works (setting aside the mixed effects aspects of a GLMM)? Do you understand how power analyses work generally? Do you understand how the effect sizes differ between studies with >2 groups vs 2 groups? It may be that some background reading will help you better situate your question. – gung - Reinstate Monica May 04 '21 at 01:18

0 Answers0