14

My question concerns priors on effect sizes, in my project the measure is Cohen's $D$. Through reading the literature, it seems vague priors are often used, such as in the well-know eight schools example of a hierarchical Bayesian meta-analysis. In the eight schools example, I have seen a vague prior used for the estimate of mu, such as $\mu_{\theta} \sim \operatorname{normal}(0, 100)$.

My discipline is psychology, where effects sizes are usually small. As such, I was considering using this prior: $\mu_{\theta} \sim \operatorname{normal}(0, .5)$. My rationale for such a tight prior is that, from my understanding of priors, I am placing a 95% prior probability that $\mu_{\theta}$ is between -1 to 1, leaving a 5 % prior probability for effects being larger that -1 or 1.

As very rarely are effects this large, is this prior justifiable?

mdewey
  • 16,541
  • 22
  • 30
  • 57
D_Williams
  • 685
  • 1
  • 5
  • 13
  • 3
    A couple of straight-forward sensitivity tests would be to use student t distributions with 4 or 7 degrees of freedom and changing the scale of the distribution. If you suspect publication bias in your sample, these sensitivity tests will not tell you a lot. You can take publication bias into account in your prior. See the work of Joachim Vandekerckhove http://www.cogsci.uci.edu/~joachim/publications.php – stijn Apr 06 '16 at 03:01

1 Answers1

2

As very rarely are effects this large, is this prior justifiable?

I think your priors are OK, as long as you can defend them with extra-statistical arguments (e.g by looking at established works in the psychological scholarly literature).

However, make sure you also perform a sensitivity analysis using less informative priors, to check whether your posterior distribution relies too heavily on your assumptions. If this is the case, with similar findings in terms of direction and magnitude of effect, then your results will appear much more robust and valid.

Giuseppe Biondi-Zoccai
  • 2,244
  • 3
  • 19
  • 48