2

What the title says!

My intuition is NO since in Bayesian statistics we typically specify the prior and likelihood, and from those two we can compute the posterior and so on. We can interpret $Y$ = data and $X$ = unknown parameters, and thus we are given prior $f_X(x)$ and posterior $f_{X|Y}(x|y)$, instead of likelihood $f_{Y|X}(y|x)$, and want to find data distribution $f_Y(y)$ without accounting for parameters $X$.

I would appreciate if someone could confirm either way, either with a description of how to compute $f_Y(y)$ if it is indeed possible, or with a simple counter-example from which I could construct two joint PDFs (or PMFs) $f(x,y)$ and $g(x,y)$ of $(X,Y)$ such that the marginal PDFs of $X$ and the conditional PDFs of $X|Y=y$ are exactly the same for $f$ and $g$, but for which the marginal PDFs of $Y$ are different for $f$ and $g$.

Thanks!

Stephan Kolassa
  • 95,027
  • 13
  • 197
  • 357
Orlando
  • 61
  • 3
  • 6
    A simple example: if the distribution of $X$ is independent of $Y$, then you certainly can't recover the distribution of $Y$ from those two distributions which you've specified. – πr8 Nov 12 '19 at 21:57

2 Answers2

2

In some cases you can. A first example is $X|Y=y \sim N(y,1)$, that is, $f_X(x)$ is the convolution between $f_Y(y)$ and a standard normal density. The relationship between the characteristic functions of $X$ and $Y$ is then $$ \varphi_X(t) = Ee^{itX} = EE(e^{itX}|Y)=E \varphi_{X|Y}(t)=Ee^{itY-t^2/2}=e^{-t^2/2}\varphi_Y(t). $$ Solving for $\varphi_Y(t)$ and using the inverse Fourier transform you have found the distribution of $Y$ (if it exists) by deconvolution.

A second example is $X|Y=y \sim N(0,y)$, that is, $f_X(x)$ is a mixture of normals with different variances. The characteristic functions are then related by $$ \phi_X(t)=E\varphi_{X|Y}(t)=E e^{-t^2Y/2}=E e^{i^2t^2Y/2}=\varphi_Y(it^2/2). $$ Solving for the characteristic function of $Y$, $$ \varphi_Y(t)=\varphi_X\big(\sqrt{t}(1-i)\big). $$ This assumes that the domain of $\varphi_X$ and $\varphi_Y$ both can be extended to the complex numbers which holds true if both $X$ and $Y$ has moment generating functions.

Jarle Tufto
  • 7,989
  • 1
  • 20
  • 36
  • Very interesting. So unfortunately there's not a straightforward answer: sometimes we can, sometimes we can't? – Orlando Nov 12 '19 at 23:35
  • 1
    Yes, if the marginal distribution of $X$ is not compatible with the distribution of $X|Y=y$, one possibility is that you end up with a mixture distribution $f_Y(y)$ having negative point masses or densities as in https://stats.stackexchange.com/a/434486/77222 and https://stats.stackexchange.com/a/332122/77222 and perhaps even stranger "pdfs/pmfs". – Jarle Tufto Nov 13 '19 at 12:08
0

Ok I figured out a simple example:

Let $X_1,X_2$ be iid $N(0,1)$ and independent of $Y$, let $Z = \begin{cases} 1, &\text{if } Y\leq 0\\ 0, &\text{if } Y > 0\end{cases}$, and $X = Z X_1 + (1-Z)X_2$.

Then, for any PDF $f_Y(y)$ such that $P(Y\leq 0) = P(Y>0) = \frac{1}{2}$, we have $X\sim N(0,1)$ and also $X|Y=y \sim N(0,1)$.

Orlando
  • 61
  • 3