15

I'm looking for an answer that would satisfy a reader who understands frequentist p-values but only understands the rudiments of Bayesian approaches to statistics.

At present google searches do not reveal a definition either on a Wikipedia page or any other commonly accepted resource.

This question seems related but isn't really since it transpired that the user was not actually calculating Bayesian p-values. However, the accepted answer links to this Gelman paper in explanation of what Bayesian p-values are.

Community
  • 1
user1205901 - Reinstate Monica
  • 11,303
  • 26
  • 77
  • 152
  • 1
    This Gelman [paper](https://projecteuclid.org/download/pdfview_1/euclid.ejs/1382448225) may also be of interest. – Sean Easter Sep 07 '15 at 02:24
  • And [this related question](http://stats.stackexchange.com/questions/148649/how-to-respond-to-reviewers-asking-for-p-values-in-bayesian-multilevel-model) – neerajt Feb 20 '16 at 01:41

2 Answers2

15

If I understand it correctly, then a Bayesian p-value is the comparison of a some metric calculated from your observed data with the same metric calculated from your simulated data (being generated with parameters drawn from the posterior distribution).

In Gelmans words: "From a Bayesian context, a posterior p-value is the probability, given the data, that a future observation is more extreme (as measured by some test variable) than the data"

For example, the number of zeros generated from a poisson based model could be such a metric or test statistic, and you could calculate how many of your simulated datasets have a larger fraction of zeros than you actually observe in your real data. The closer this value to 0.5, the better the values calculated from your simulated data distribute around the real observation.

Blundering Ecologist
  • 243
  • 2
  • 14
grg rsr
  • 288
  • 3
  • 9
2

Bayesian p-values are normally used when one would like to check how a model fits the data. That is, given a model $M$ we wish to examine how well it fits the observed data $x_{obs}$ based on a statistic $T$, which measures the goodness of fit of data and model. For this, suppose we have a model $M$ with probability density function $f(x|\theta)$ and with prior $g(\theta)$. Then, one can define the prior predictive p-value or tail area under the predictive distribution through the expression

$$ p = P(T(x)\geq T(x_{obs})|M) = \int_{T(x)\geq T(x_{obs})}h(x)dx, $$

where

$$h(x)dx = \int f(x|\theta)g(\theta)d\theta$$

is the prior predictive density.

Notice that this approach may be influenced by the choice of the prior (for an example, see pg.180 of [1]). For this reason, the posterior predictive p-value was introduced. Now, consider that the prior depends on the observed data $g(\theta|x_{obs})$, thus,

$$ h(x|x_{obs}) = \int f(x|\theta)g(\theta|x_{obs})d\theta. $$

However, this approach presents two disadvantages. First, we're considering a double use of the data (for the definiton of $h(x)$ and $p$). Second, for larger sample sizes, the posterior distribution of $\theta$ concentrates at the Maximum Likelihood Estimate of $\theta$ (the frequentist or classical approach).

To overcome this, the conditional predictive distribution was introduced. Consider a statistic $U$ that does not involve the statistic $T$. Then, the conditional predictive p-value is

$$ p_{c} = P^{h(\cdot|u_{obs})}(T(x)\geq T(x_{obs})|M) = \int_{T(x)\geq T(x_{obs})} h(t|u_{obs}) dt, $$

where $h(t|u_{obs})$ is the conditional predictive density of $T$ given $U$ and $u_{obs}$ is $U(x_{obs})$.

Additionally, one could consider the partial posterior predictive p-value with the advantage of not requiring a choice for the statistic $U$, see pg.184 of [1] for more details.

[1] Ghosh, Jayanta; Delampady, Mohan; Samanta, Tapas. An Introduction to Bayesian Analysis:Theory and Methods. Springer, 2006.

Miguel Trejo
  • 261
  • 1
  • 3