5

I have created a d-efficient fractional factorial design of 48 combinations from a total of 192 possible combinations (4x2x2x3x2x2).

For the experiment, I plan to have 4 runs in 12 blocks and 40 individuals for each block completing the 4 runs. Each individual sees 4 text vignettes and gives a 10 point rating for each. In each vignette, the 6 dimensions vary.

So I would have 480 individuals participating and 1920 observations in total.

To my knowledge, this repeated measurement of individuals (4 times) needs to be accounted for. Is a mixed model an appropriate way to do so? I basically would have data from runs nested in individuals. I have heard some arguments, that a mixed model is not appropriate for this design, since there is no theoretically interesting variance at the lower level, since the treatment combinations are essentially randomized?

The main research interest is the marginal effect of the 6 dimensions.

But to be honest, I am still very confused about this. I know that it is possible to estimate a mixed model with this data, but I am unsure if its the right thing to do given the design.

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
T-Porsch
  • 63
  • 7
  • Can you give a reference for that claim? – kjetil b halvorsen Feb 20 '20 at 19:11
  • I wish I could. It's basically just word of mouth. But I would be interested in literature recommendations for analysis of factorial experiments where one "treatment" consists of many different treatment dimensions. – T-Porsch Feb 20 '20 at 19:29
  • It would help if you could give some more context, what does all this variables represent? What is your response variable? How do you define the blocking ? ... But, you have multiple measurements for the same individuals, that points to repeated measurements, which it is natural to analyze with a mixed model. – kjetil b halvorsen Feb 20 '20 at 19:41
  • The response variable is a 10 point rating. The context is relatively straigthforward: Individuals receive a short text vignette where all 6 dimensions vary. I added this information in the op aswell. – T-Porsch Feb 21 '20 at 10:09

1 Answers1

5

I have heard some arguments, that a mixed model is not appropriate for this design, since there is no theoretically interesting variance at the lower level, since the treatment combinations are essentially randomized?

I would be interested to know where you read those arguments.

To my knowledge, this repeated measurement of individuals (4 times) needs to be accounted for. Is a mixed model an appropriate way to do so?

I don't see any reason why you shouldn't use a mixed effects model. You have repeated measurements, and fitting random intercepts for each level of clustering is a good way to account for the non-independence of observations within each cluster.

It is not completely clear from the question what the hierarchy of nested factors are, but it sounds a lot like individuals are nested within blocks. That is each subject is measured 4 times (4 runs per subject) and each subject belongs to one of 12 blocks. Thus your model could look something like:

outcome ~ treatments + (1 | block/subject)

If the allocation to blocks is conmpletely random and the randomization was successful then you may not need block in the random structure.

Robert Long
  • 53,316
  • 10
  • 84
  • 148
  • Thanks for your comment. I think one argument is that the variance at level 1 (the individual ratings) is fully explained by the combination of all treatments and treatment interactions. This comes mainly from the potential outcomes framework. So I'm not really interested in ATEs of different dimension combinations, but in the marginal effect of dimensions. – T-Porsch Feb 21 '20 at 11:06
  • @T-Porsch do you have a reference for that claim? It might be true in some situations but I find it hard to see that it would be true in general. In any case, if it is true in a specific case then the model will estimate zero for that variance component, so in the first instance I would use a mixed model and see where it takes you, since if it is not zero and you fail to account for it then you may obtain biased results. – Robert Long Feb 21 '20 at 11:24
  • It's mostly from the literature on Average Marginal Component Effects (Imai and others). – T-Porsch Feb 22 '20 at 20:16