3

We have conducted a meta-analysis and meta-analysed the effect sizes. This is a meta-analysis of 2x2 experimental designs, so we meta-analysed two main effect effect sizes and one interaction. Now we would like to illustrate the findings by creating a visualisation where the four meta-analysed means are plotted (allowing us to show the effect sizes as well), in a plot similar to this:

ggplot(data.frame(x=factor(c(0, 0, 1, 1)),
                  y=c(0.9, 1.1, 1.8, 0.3),
                  z=factor(c(0,1,0,1))),
       aes(x=x,y=y, group=z, color=z)) +
  geom_point(size=2) +
  geom_line(size=1) +
  theme_minimal();

Line plot of four means in a 2x2 experiment

We have standard deviations and means for each of the four cells, so we can standardize those (e.g. using the grand mean and the standard deviation of the total sample) to get four means that each represent the deviation from the grand mean in standard deviations.

For example, for one study, we have:

means <- c(0.32, 0.28, 0.36, 0.54);
sds <- c(0.41, 0.41, 0.41, 0.58);
ns <- c(36, 36, 36, 36);

grandMean <- sum((means*ns)/sum(ns));
grandSD <- sqrt(sum(sds^2 * (ns-1)) / (sum(ns) - 4));

zMeans <- (means - grandMean) / grandSD;

And here we go:

print(zMeans);
[1] -0.11996986 -0.20722067 -0.03271905  0.35990959

But, how can a series of such means be meta-analysed?

Is it so simple so as to simply consider them means from the same study? So could we just compute new means, weighing each mean using the sample size of that cell in that study?

On the one hand, this seems to be what we're doing. We have means from the same conditions but from different samples - not too different from when you aggregate means of different conditions but from the same sample like you do when you compute the grand mean.

I've found this post, but that package doesn't seem to provide an answer either.

I've used metafor to do meta-analyses of proportions (i.e. prevalences) before, and there were dedicated methods for that. But I can't seem to find anything for this situation, perhaps because it is as simple as it seems? Or because I'm missing something.

I'd be grateful for any pointers! Or simply the confirmation that yes, it's this simple :-)

Matherion
  • 477
  • 4
  • 13
  • I am not sure I fully understand this but if you analyse each study using regression you would get a coefficient for X, Y and X:Y and their variance covariance matrix which you could then input into a function for multivariate meta-analysis like rma.mv. If you do not get a response here as questions about R code often get closed you might like to try the recently established mailing list for R meta-analysis queries https://stat.ethz.ch/mailman/listinfo/r-sig-meta-analysis You do need to register before posting – mdewey Aug 17 '17 at 08:24
  • Well, I could remove the examples in R - they're illustrative, but nothing more. The question is: if you have N means from different studies, how can you 'average' these over studies? (So, not effect sizes: simple univariate means.) In any case, thank you for your response, I will definitely subscribe to that mailing list!!! – Matherion Aug 17 '17 at 08:48

2 Answers2

2

@Matherion, the discrepancies you are finding are because weights in meta-analysis are not calculated based on the number of participants, but based on the reciprocal of the variance. Two models are commonly used: 1) the fixed-effects model, where the only source of variance is assumed to be sampling error, and the weights are thus 1/vi (vi is obtained from escalc in metafor). 2) the random-effects model, which assumes that true effect sizes follow a distribution with its own variance, the between-studies variance tau^2, and the weights are given by 1/(vi+tau^2).

The random effects model is generally more appropriate, unless these studies are exact replications. By default metafor conducts a random-effects meta-analysis.

Caspar van Lissa
  • 121
  • 5
  • Thank you for the explanation. I didn't know this was also true for means. But I guess it makes sense - they're all just parameters from sampling distributions, after all, doesn't matter whether they concern univariate or bivariate (or, I suppose, multivariate) measures. – Matherion Aug 17 '17 at 11:28
1

Based on @mdewey's answer, I found out that (of course) metafor does this: section "Measures for Quantitative Variables" in the manual page for escalc explains how means can be specified. Using this, I can confirm that the 'basic' approach I mentioned in the question is invalid:

rma(escalc(measure="MN", mi=c(1,2,3), sdi=c(1,1,1), ni=c(10,10,10)))

This yields $2$ as result. This is consistent with what you'd expect: if you compute the grand mean for $1$, $2$, and $3$ with three samples with the same standard deviation and sample size, you also get $2$.

Now, if we'd change the size of the first group to $30$, if you compute the grand mean, you get:

$$\frac{30 * 1 + 10 * 2 + 10 * 1}{30 + 10 + 10} = 1.6$$

But, entering this into rma and escalc yields $1.9778$:

rma(escalc(measure="MN", mi=c(1,2,3), sdi=c(1,1,1), ni=c(30,10,10)))

Apparently, indeed, the underlying calculation is more complicated.

So if anybody else runs into this: no, you can't just compute the 'grand mean'; and metafor allows you to meta-analyse the raw means.

Matherion
  • 477
  • 4
  • 13