Bayesian statistical conclusions: we implicitly condition on the known values of any covariates, $x$

Question

My Bayesian data analysis textbook says the following:

Bayesian statistical conclusions about a parameter $\theta$, or unobserved data $\tilde{y}$, are made in terms of probability statements. These probability statements are conditional on the observed value of $y$, and in our notation are written simply as $p(\theta|y)$ or $p(\tilde{y}|y)$. We also implicitly condition on the known values of any covariates, $x$.

Page 6, Bayesian Data Analysis, Third Edition, by Gelman et al.

I'm wondering what is meant by this last part:

We also implicitly condition on the known values of any covariates, $x$.

I'm aware that $x$ are the explanatory variables (also known as covariates or predictors), but how do we "implicitly condition on them"? What is meant by this?

I would greatly appreciate it if people could please take the time to explain this.

Answer 1

To implicitly condition on a variable simply means that we condition on it, but we do not state it as a conditioning variable in the probability statement (i.e., the conditioning is implicit, not explicit). This is usually done for reasons of brevity, particularly if you are making a whole lot of statements where you are always conditioning on $x$, and it would be more succinct just to omit this from your notation entirely, to avoid burdening the reader.

So what Gelman et al are saying is that they will continue to use notation like $p(\theta|y)$ and $p(\tilde{y}|y)$ that does not have $x$ stated as a conditioning variable, but their working can also be interpreted as if $x$ was an implicit conditioning variable in every statement. So when they use these functions in statements, they are really referring to $p(\theta|y,x)$ and $p(\tilde{y}|y,x)$ respectively.

In regard to this issue, it is also worth noting that many theories of probability regard all probability to be conditional on implicit information. This idea is most famously associated with the axiomatic approach of the mathematician Alfréd Rényi (see e.g., Kaminski 1984). Rényi argued that every probability measure must be interpreted as being conditional on some underlying information, and that reference to marginal probabilities was merely a reference to probability where the underlying conditions are implicit.

Answer 2

In addition to the other excellent answer, here I will try to make a more explicit argument. Making the argument explicit is helpful in understanding its underlying assumptions, so we can judge when to use the argument, and when to avoid it. This will be a bayesian version of the argument made in What is the difference between conditioning on regressors vs. treating them as fixed?, and I will use notation from there.

So assume we are interested in some regression-like model for random vector $(X, Y)$, with joint density $f(y,x \mid \theta,\psi)$ which can be factored as $$ f_\theta(y\mid x)\cdot f_\psi(x) $$ where $\theta$ is an unknown parameter in the conditional distribution of $Y$ given $X$ (the regression model), while $\psi$ is an unknown parameter in the marginal distribution of $X$. We assume the focus of interest is in the regression relationship, so $\theta$ is the focus or interest parameter, while $\psi$ is an incidental parameter.

If now the prior distribution factorizes in the same way, that is $$ \pi(\theta,\psi) = \pi_1(\theta)\cdot \pi_2(\psi) $$ then after some manipulation we find that $$ \pi(\theta,\psi \mid y,x) = \pi_1(\theta \mid y,x)\cdot \pi_2(\psi\mid x) $$ where $$ \pi_1(\theta\mid y,x)=\frac{f_\theta(y\mid x) \pi_1(\theta)}{\int f_\theta(y\mid x) \pi_1(\theta)\; d\theta} \\ \pi_2(\psi \mid x) = \frac{f_\psi(x) \pi_2(\psi)}{\int f_\psi(x) \pi_2(\psi)\; d\psi} $$ So, under our assumptions, the posterior distribution factors in the same way as the prior, and so if our only interest is in the regression relationship (thus in $\theta$), we do not need to model $f_\psi(x)$ at all, so can condition on $x$.

This framework also makes it easy to see when such conditioning is problematic, an obvious example is when we include lagged responses as predictors. Another case is with omitted variables, in a regression model omitted variables will implicitely be part of the error term, and so if an omitted variable is correlated with other predictors, that induces correlations between $X$ and the error term in the regression, destroying the factorization.