7

Reviewer asked me why I use meta-regression as a way how to deal with heterogeneity among effect sizes instead of conducting stratified meta-analysis.

I tried to google "stratified meta-analysis" and probably the most useful explanation was:

Stratification is an effective way to deal with inherent differences among studies and to improve the quality and usefulness of the conclusions. An added advantage to stratification is that insight can be gained by investigating discrepancies among strata. There are many ways to create coherent subgroups of studies. For example, studies can be stratified according to their “quality,” assigned by certain scoring systems. Commonly used systems award points on the basis of how patients were selected and randomized, the type of blinding, the dropout rate, the outcome measurement, and the type of analysis (eg, intention-to-treat).

Walker, E., Hernandez, A. V., & Kattan, M. W. (2008). Meta-analysis: Its strengths and limitations. Cleveland Clinic Journal of Medicine, 75(6), 431–439.

From what I understand, the I should make some scoring system for my sample of studies, and use that score as a "weight" in my meta-analytic model? I do not like this idea. It seems to my more less objective than meta-regression mainly because I have no criteria in my studies to make the score. (I am doing meta-analysis of ecological studies.)

May I use this as an argument in response that stratified meta-analysis will be less objective in my case?

Andrew M
  • 2,696
  • 14
  • 25
Ladislav Naďo
  • 2,202
  • 4
  • 21
  • 45
  • 2
    Are you asking about the distinction between stratifying on some categorical variable versus including said categorical variable as a factor in a (mixed-effects) meta-regression model? – Wolfgang Dec 19 '14 at 14:40
  • Hi @Wolfgang! Thank you for response. Here I add the original comment: "Given the presence of significant heterogeneity across studies, the authors elected to add selected predictors into the random effects model. Why didn't the authors instead stratify their meta-analyses on some of these factors?". My model tries to explain heterogeneity among effect sizes with 4 continuous predictors (none of them is categorical) which is pretty successful (more than half of variability explained). – Ladislav Naďo Dec 19 '14 at 15:18

1 Answers1

6

Here are some suggestions for how you could respond:

  1. Given that your predictors are continuous (and artificially categorizing predictors is usually frowned upon), meta-regression seems like the right approach (and in fact, meta-regression can also deal with categorical predictors just as well as stratifying on them, so why bother?).
  2. If I understand you correctly, you entered those 4 predictors simultaneously into the model. Stratifying would either imply examining one (artificially categorized) predictor at a time (which does not consider heterogeneity that may be better accounted for by other predictors or potential confounding between the predictors) or if one were to start stratifying on combinations of predictors, the subgroups will start to get quite small. That doesn't seem like a good idea (see also the next point).
  3. How well the amount of heterogeneity in a random-effects model (or the amount of residual heterogeneity in a mixed-effects meta-regression model) is estimated depends to a great extent on the number of studies. Stratifying will lead to smaller subsets with poorer estimates of the amount of heterogeneity.

I actually discuss these issues in this article:

Viechtbauer, W. (2007). Accounting for heterogeneity via random-effects models and moderator analyses in meta-analysis. Zeitschrift für Psychologie / Journal of Psychology, 215(2), 104-121.

If you are interested and cannot get hold of a copy of the article (it's in a German journal; but the article is in English), feel free to send me an e-mail (you'll find my website linked to from my profile; e-mail address can be found there).

Wolfgang
  • 15,542
  • 1
  • 47
  • 74