Use of prior and posterior predictive distributions?

Question

I understand the prior and posterior distributions and I have read what the prior and posterior predictive distributions are.

However, I don't really see the point of knowing them.

Knowing more things wouldn't hurt, but I just want to understand the reason why I need to know them.

Answer 1

Some uses of the posterior predictive:

• Simulating future data based on your model assumptions and data observed to this point. This is useful for predictions, forecasting, etc.
• Model checking via posterior predictive checking. Some comments have directed you to Bayesian Data Analysis, and its author has made a relevant chapter available. Tim's answer to this question should also prove helpful.

I've less help to offer on the prior predictive. I've found it useful as a sort of summary check on my combined priors: It can serve as an intuitive summary of your ultimate prior assumptions on expected data.

In a similar vein, some view it as a tool to arrive at informative priors. Consider this correspondence shared on Andrew Gelman's blog:

I don’t ever see parameters. Some model have few and some have hundreds. Instead, I see data. So I don’t know how to have an opinion on parameters themselves. Rather I think it far more natural to have opinions on the behavior of models. The prior predictive density is a good and sensible notion.

A further post continues:

The goal is to use the “black box” of the prior predictive density and the prior conditional density (the conditional in particular since you can look at model behaviour in a dynamic, scenario based setting) to inform us about how the informative priors should be constrained.

Put another way, if you're struggling to set prior parameters, you may find it sensible to examine those parameters' consequences on expected data. Doing so requires the prior predictive.

Answer 2

Let's you denote your data as $X$ and imagine that you have some probabilistic model that describes your data in terms of the likelihood of observing your data given some parameter $\theta$. The parameter $\theta$ is unknown and is to be estimated from your data. To estimate your parameter you could use many different approaches, e.g. use maximum likelihood estimation to find such value of $\theta$ that maximizes the likelihood, or use Bayesian approach. In Bayesian approach to estimate the parameter we need one more thing, a prior distribution for $\theta$. If you take those things together, you can use Bayes theorem to obtain the posterior distribution of $\theta$ (i.e. your estimate):

$$ \underbrace{p(\theta|X)}_\text{posterior} \propto \underbrace{p(X|\theta)}_\text{likelihood} \, \underbrace{p(\theta)}_\text{prior} $$

So to catch up:

• prior is the distribution of $\theta$ that is assumed by you before seeing the data. You do not "know" it, it is something that you assume.
• likelihood is the conditional distribution of data given the prior, it defines your model,
• posterior is the "estimated" distribution of parameter $\theta$ given data and the prior,
• posterior predictive distribution is the distribution of data that is "predicted" by your model given your data and the prior, it describes your predictions from the model.

For example, in the classical beta-binomial model (see beta-binomial for multiple examples and more details) we have

$$ X \mid \theta \sim \mathrm{Binom}(n, \theta) \\ \theta \sim \mathrm{Beta}(\alpha, \beta) $$

so binomial distribution parametrized by $\theta$ is our likelihood, beta distribution with hyperparameters $\alpha,\beta$ is our prior and by using conjugacy we can obtain a closed-form solution for posterior

$$ \theta \mid X \sim \mathrm{Beta}(x+\alpha, n-x+\beta) $$

and posterior predictive distribution

$$ \tilde X \mid X,\theta \sim \mathrm{BetaBinom}(n, x+\alpha, n-x+\beta) $$

where $\tilde X$ are the $X$'s predicted by our model.

If this all is still unclear for you you can check the multiple threads tagged as bayesian on this site, or multiple handbooks on this topic.