Power of Meta Regression

Question

Why it is said that Meta Regression analyses have low power? & How to calculate the same?

Answer 1

The N for the main effect in a meta-analysis is something close to the number of people in all of the studies. That's a lot of people, hence a lot of power.

For a meta-regression, the N is much closer to the number of studies, which is a much smaller number. The actual hit in power that you take is a function of the degree of heterogeneity. It's equivalent to looking at a multilevel model (or clustered trial) with a very high ICC.

Here's a paper where we discuss some of the issues: Risk of bias: a simulation study of power to detect study-level moderator effects in meta-analysis. The graphs show the amount of power, for different moderator effects, numbers of studies, sample sizes, and heterogeneity. With a high value of tau^2, you need a huge number of large studies, and a large moderator effect, to have decent power.

Answer 2

This is kind of a dogma for meta-analysts, yet there is not much out there.

I have researched Google, Google Scholar and PubMed, looking for power, metaregression, meta-regression.

These are the items I have found which could be of use:

Thompson and Higgins, Statistics in Medicine 2002

Higgins and Thompson, Statistics in Medicine 2004

Borenstein et al, Introduction to Meta-Analysis 2009

Whittington, Statalist 2009

Laurin, University of Notre Dame 2014

Lopez-Lopez et al, British Journal of Mathematical and Statistical Psychology 2014

This passage from Borenstein et al, Introduction to Meta-Analysis 2009, is pertinent to your question, despite being somewhat generic:

Power depends on the size of the effect and the precision with which we measure the effect... For meta-regression this means that power will increase as the magnitude of the relationship between the covariate and effect size increases, and/or the precision of the estimate increases... a key factor driving the precision of the estimate will be the total number of individual subjects across all studies and (for random effects) the total number of studies. While there is a general perception that power for testing the main effect is consistently high in meta-analysis, this perception is not correct ... and certainly does not extend to tests of subgroup differences or to meta-regression. The failure to find a statistically significant p-value when comparing subgroups or in meta-regression could mean that the effect (if any) is quite small, but could also mean that the analysis had poor power to detect even a large effect. One should never use a nonsignificant finding to conclude that the true means in subgroups are the same, or that a covariate is not related to effect size.

My personal take is that many factors play a role, and so computing power/minimal sample size is not that straightforward. However, we may simply consider meta-regression as a type of weighted regression or mixed model, and so power considerations pertinent to them should be valid.

Answer 3

People usually do a power calculation before starting work to establish the sample size but in meta-analysis the sample size is seldom something you can choose. After the event all the information about precision is contained in the confidence intervals for your coefficients so power is now irrelevant.