0

Suppose I have a posterior distribution, $p(\theta \mid y)$, where $y$ was my data and $\theta$ is a random variable with some prior distribution. If I specify a one-to-one transformation $\phi = g(\theta)$, how does the Jacobian look like? Would it be:

$$ p(g(\theta)\mid y) = p(\theta|y)|g(\theta)|^{-1} $$ ?

I am ultimately trying to prove that the MAP is not invariant to one-to-tone transformations and was hoping that the $|g(\theta)|^{-1}$ would be the key. However, I am not sure how to conduct function transformations of a random variable from a conditional density. Can someone offer tips? Thanks!

user321627
  • 2,511
  • 3
  • 13
  • 49
  • Just a note: the MAP *is* invariant to 1-to-1 transformations. The easiest way for me to see this (which might not be generally the shortest path...) is that any posterior can be seen as proportional to some other likelihood function in which the prior was actually a part of the likelihood itself, and the MLE is invariant to transformations. Therefore, the MAP is invariant to transformations. – Cliff AB Feb 24 '18 at 03:25
  • 1
    I was under the impression the MAP is not generally invariant to transformations, from this link: https://math.stackexchange.com/questions/2318467/why-are-map-estimators-not-invariant-under-reparameterization-while-mles-are. Are you saying that only in the case of 1-to-1 transforms it is? – user321627 Feb 24 '18 at 03:35
  • Oh, that's right! The issue here is that when you reparameterize, you also change the PDF of your prior, so while this would still be equivalent to *some* likelihood function, it's no longer the same likelihood and so can have a different mode. Now I will have to think about why this wouldn't happen with a hierarchical model fit with MLE at all levels... – Cliff AB Feb 24 '18 at 05:56
  • 2
    @CliffAB: the likelihood function is invariant under a change of parameterisation (and hence the MLE is equivariant). The reason why the MAP is not equivariant is that the Jacobian of the transform impacts the location of the maximum of the posterior density. – Xi'an Feb 26 '18 at 10:34

1 Answers1

2

Just take the conditional distribution as the distribution of interest:

$$ q(\theta) = p(\theta \vert y),$$

since $y$ is "given". This is justified by the definition of the conditional density/distribution. The rest is the usual change of variable, where you include the Jacobian. So, the distribution of $\phi = g(\theta)$ is:

$$q(g^{-1}(\theta))\vert g'(\theta)\vert^{-1}.$$

https://onlinecourses.science.psu.edu/stat414/node/157

https://en.wikipedia.org/wiki/Conditional_probability_distribution

Condor
  • 36
  • 1