Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [meta-analysis]

Methods focused on contrasting and combining results from different studies, in the hope of increasing precision and external validity.

Meta-analysis refers to methods focused on contrasting and combining results from different studies, in the hope of identifying patterns among study results, sources of disagreement among those results, or other interesting relationships that may come to light in the context of multiple studies.

Some aspects of research that can be meta-analyzed include (credit due to this answer):

  • Heterogeneity of results
    • Moderators (see also )
      • Construct measurement and analytic methods
        • Study quality
      • Population
    • Publication bias
      • Fail safe N – number of null results necessary to reduce an effect size to insignificance

References (credit due to this question: Looking for good introductory treatment of meta-analysis)

- Anglim, J. (2009, December 7). Meta-analysis: Tips, resources, and software. Jeromy Anglim's Blog: Psychology and Statistics. Retrieved from http://jeromyanglim.blogspot.com/2009/12/meta-analysis-tips-resources-and.html.
- Borenstein, M., Hedges, L. V., Higgins, J. P., & Rothstein, H. R. (2011). Introduction to meta-analysis. John Wiley & Sons. Sample chapters available at http://www.meta-analysis-workshops.com/download/bookChapterSample.pdf.
- Dallal, G. E. (2003). Meta analysis. Retrieved from http://www.jerrydallal.com/LHSP/meta.htm.
- DeCoster, J. (2004). Meta-analysis notes. Retrieved February 22, 2014 from http://www.stat-help.com/meta.pdf.
- Egger, M., Smith, G. D., & Altman, D. (Eds.). (2008). Systematic reviews in health care: meta-analysis in context. John Wiley & Sons.
- Harrison, F. (2011). Getting started with meta‐analysis. Methods in Ecology and Evolution, 2(1), 1–10.
- Persuad, R., & Evans, S. (1996). Misleading meta-analysis: "Fail safe N" is a useful mathematical measure of the stability of results. British Medical Journal, 312, 125. Retrieved from http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2349768/pdf/bmj00524-0065a.pdf.
- Wolf, F. M. (1986). Meta-analysis: Quantitative methods for research synthesis (Vol. 59). Sage.

1213 questions
32
votes
3 answers

Why does finding small effects in large studies indicate publication bias?

Several methodological papers (e.g. Egger et al 1997a, 1997b) discuss publication bias as revealed by meta-analyses, using funnel plots such as the one below. The 1997b paper goes on to say that "if publication bias is present, it is expected…
z8080
  • 1,598
  • 1
  • 19
  • 38
30
votes
3 answers

Can a meta-analysis of studies which are all "not statistically signficant" lead to a "significant" conclusion?

A meta-analysis includes a bunch of studies, all of which reported a P value greater than 0.05. Is it possible for the overall meta-analysis to report a P value less than 0.05? Under what circumstances? (I am pretty sure the answer is yes, but I'd…
Harvey Motulsky
  • 14,903
  • 11
  • 51
  • 98
28
votes
3 answers

What causes the discontinuity in the distribution of published p-values at p < .05?

In a recent paper , Masicampo and Lalande (M-L) collected a large number of p-values published in many different studies. They observed a curious jump in the histogram of the p-values right at the canonical critical level of 5%. There is a nice…
Zen
  • 21,786
  • 3
  • 72
  • 114
24
votes
2 answers

Statistical forensics: Benford and beyond

What broad methods are there to detect fraud, anomalies, fudging, etc. in scientific works produced by a third party? (I was motivated to ask this by the recent Marc Hauser affair.) Usually for election and accounting fraud, some variant of…
shabbychef
  • 10,388
  • 7
  • 50
  • 93
22
votes
2 answers

Combining information from multiple studies to estimate the mean and variance of normally distributed data - Bayesian vs meta-analytic approaches

I have reviewed a set of papers, each reporting the observed mean and SD of a measurement of $X$ in its respective sample of known size, $n$. I want to make the best possible guess about the likely distribution of the same measure in a new study…
D L Dahly
  • 3,663
  • 1
  • 24
  • 51
20
votes
2 answers

Multiple imputation for outcome variables

I've got a dataset on agricultural trials. My response variable is a response ratio: log(treatment/control). I'm interested in what mediates the difference, so I'm running RE meta-regressions (unweighted, because is seems pretty clear that effect…
generic_user
  • 11,981
  • 8
  • 40
  • 63
18
votes
2 answers

Quality assurance and quality control (QA/QC) guidelines for a database

Background I am overseeing the input of data from primary literature into a database. The data entry process is error prone, particularly because users must interpret experimental design, extract data from graphics and tables, and transform results…
David LeBauer
  • 7,060
  • 6
  • 44
  • 89
18
votes
4 answers

Combining two confidence intervals/point estimates

Suppose one has two independent samples from the same population, and different methods were used on the two samples to derive point estimate and confidence intervals. In trivial cases a sensible person would just pool the two samples and use one…
user1600
  • 181
  • 1
  • 1
  • 5
18
votes
5 answers

Does Bayesian statistics make meta-analysis obsolete?

I'm just wondering if Bayesian statistics would be applied consequently from the first study to the last if this makes a meta-analysis obsolete. For example, let's assume 20 studies which have been done at different timepoints. The estimate or…
giordano
  • 829
  • 8
  • 19
15
votes
3 answers

Pros and cons of meta-analyses

I have been considering doing some meta-analysis for a particular field of study in evolution, but before I go any further I would like to know; what are the positives and negatives of the process? For example, no need for a practical experiment is…
rg255
  • 752
  • 2
  • 8
  • 27
14
votes
1 answer

Specifying prior for effect size in meta-analysis

My question concerns priors on effect sizes, in my project the measure is Cohen's $D$. Through reading the literature, it seems vague priors are often used, such as in the well-know eight schools example of a hierarchical Bayesian meta-analysis. In…
D_Williams
  • 685
  • 1
  • 5
  • 13
14
votes
4 answers

Looking for good introductory treatment of meta-analysis

A (non-statistician) colleague has been encountering meta-analyses in papers he reviews for medical journals and is looking for a good introductory level treatment so he can educate himself. Any recommendations? Favorites? Books, monographs,…
whuber
  • 281,159
  • 54
  • 637
  • 1,101
13
votes
1 answer

Is meta-analysis of odds ratios essentially hopeless?

In a recent paper Norton et al. (2018)$^{[2]}$ state that Different odds ratios from the same study cannot be compared when the statistical models that result in odds ratio estimates have different explanatory variables because each model has a…
COOLSerdash
  • 25,317
  • 8
  • 73
  • 123
13
votes
2 answers

Making a Bayesian prior from a frequentist result

How should one go about turning a frequentist result into a Bayesian prior? Consider the following pretty generic scenario: An experiment was conducted in the past and a result on some parameter $\phi$ was measured. The analysis was done with a…
bill_e
  • 2,681
  • 1
  • 19
  • 33
13
votes
2 answers

Without multi-level modeling, how to handle within-study replication in a meta-analysis, where the study is the unit of replication?

Description of the study: I have observed a common error among meta-analyses, with regard to handling of within-study replication. It is not clear to me if the error invalidates the studies when assumptions are stated. However, as I understand it,…
David LeBauer
  • 7,060
  • 6
  • 44
  • 89
1
2 3
…
80 81