Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [publication-bias]

A bias in what academic research is more likely to be published; e.g. findings with significant outcomes are more likely to be published than the rest.

The term publication bias is used to describe the situation where the chances of a study being published depend on its results. Statistical attempts to detect it usually rely on detecting small study bias where it is hypothesised that large studies are more likely to be published irrespective of outcome whereas small ones may be affected. Techniques used include funnel plots and various regression tests of which the best know are due to Egger and to Begg.

The wikipedia article https://en.wikipedia.org/wiki/Publication_bias has more detail and some references.

39 questions
45
votes
5 answers

Statistics published in academic papers

I read a lot of evolutionary/ecological academic papers, sometimes with the specific aim of seeing how statistics are being used 'in the real world' outside of the textbook. I normally take the statistics in papers as gospel and use the papers to…
luciano
  • 12,197
  • 30
  • 87
  • 119
32
votes
3 answers

Why does finding small effects in large studies indicate publication bias?

Several methodological papers (e.g. Egger et al 1997a, 1997b) discuss publication bias as revealed by meta-analyses, using funnel plots such as the one below. The 1997b paper goes on to say that "if publication bias is present, it is expected…
z8080
  • 1,598
  • 1
  • 19
  • 38
32
votes
5 answers

How should an individual researcher think about the false discovery rate?

I've been trying to wrap my head around how the False Discovery Rate (FDR) should inform the conclusions of the individual researcher. For example, if your study is underpowered, should you discount your results even if they're significant at…
Richard Border
  • 1,128
  • 9
  • 26
12
votes
3 answers

Is publishing work based on post hoc analysis problematic?

Nowadays, in the scientific domain, increasingly, people in the top position argue against doing post-hoc analysis and they advise against collecting a bunch of data and just make up a story after data collection and report the significant findings.…
JetLag
  • 1,105
  • 2
  • 9
  • 22
12
votes
1 answer

Metafor package: bias and sensitivity diagnostics

I am conducting a multi-level meta-analysis that includes some articles with multiple outcomes. Therefore I am using the rma.mv() function. Example code: test.main = rma.mv(yi,vi,random = ~1|ID, data = data) I have two questions: I read in a…
chabeck
  • 336
  • 2
  • 10
11
votes
1 answer

Alternative funnel plot, without using standard error (SE)

Before submission of my meta-analysis I want to make a funnel plot to test for heterogeneity and publication bias. I have the pooled effect size and the effect sizes from each study, that take values from -1 to +1. I have the sample sizes n1, n2 for…
Staty Despair
  • 359
  • 1
  • 4
  • 13
10
votes
1 answer

Torn between PET-PEESE and multilevel approaches to meta-analysis: is there a happy medium?

I am currently working on a meta-analysis, for which I need to analyze multiple effect sizes nested within samples. I am partial to Cheung's (2014) three-level meta-analysis approach to meta-analyzing dependent effect sizes, as opposed to some of…
jsakaluk
  • 5,006
  • 1
  • 20
  • 45
9
votes
2 answers

Are most published correlations in social sciences untrustworthy and what is to be done about it?

Despite the important but smacking of "gotcha"-istic efforts by individuals to reveal the practices of predatory journals, a greater and more fundamental threat looms in the shadows of social science research (though there are certainly multiple…
Matt Barstead
  • 2,014
  • 8
  • 19
6
votes
1 answer

Testing for publication bias in meta-analysis when effect is raw mean

I am conducting a multi-level meta-analysis using rma.mv in the metafor package in R. The effect size measure I am working with is the raw mean (measure="MN" in metafor). The sampling variance (vi) for each datum is calculated as sd^2/n. I would…
chabeck
  • 336
  • 2
  • 10
5
votes
1 answer

What kind of cut-offs are used for publication bias tests in meta-analysis?

I have performed meta-analysis and included four different tests for publication bias in the analysis - classic fail-safe N, Orwin's fail-safe N, Duval and Tweedie's trim and fill as well as funnel plots. The tests did not seem to indicate…
Annie
  • 53
  • 3
4
votes
2 answers

Publication bias in meta-analysis of diagnostic accuracy in R?

Are there any implemented methods in R that could be used to assess publication bias in a meta-analysis of diagnostic accuracy (Egger's test, funnel plots)? One way would be to change sensitivites and specifities to log diagnostic odds ratios…
jokel
  • 2,403
  • 4
  • 32
  • 40
4
votes
0 answers

Are there extensions of publication-bias tests for Network Meta-Analysis?

In pairwise meta-analysis, it is common to check for publication bias. In particular, there are multiple tests for funnel-plot asymmetry (see here for Cochrane's recommendations about that:…
Federico Tedeschi
  • 814
  • 5
  • 17
4
votes
2 answers

publication bias in meta-analysis of diagnostic tests

How can investigate publication bias for diagnostic tests in meta-analysis with R? I saw this topic "Publication bias in meta-analysis of diagnostic accuracy in R?" but there is no answer yet!
Bernard
  • 363
  • 1
  • 3
  • 10
3
votes
1 answer

Alternative Egger's test, without using standard error

A simplified regression equation $ES=\frac{a+b}{n_1+n_2}$ has been suggested as an alternative to Egger's regression equation $\frac{ES}{SE}=\frac{a+b}{SE}$, where ES=Effect Size, $n_1$=sample size of the patients, $n_2$=sample size of the controls,…
Staty Despair
  • 359
  • 1
  • 4
  • 13
3
votes
2 answers

Egger's test in SPSS

How can I conduct an Egger's test using SPSS17? For each study included in the meta-analysis I know effect size and sample size of patients and controls groups.
Staty Despair
  • 359
  • 1
  • 4
  • 13
1
2 3