1

I'm using a MCMC algorithm. The proposal is, due to lack of information on my part, a multivariate T-Student distribution, i.e. $\theta \sim \mathcal{MT}(\mu, \Sigma)$. However, some of the components of $\theta$ are restricted to a specific interval. To make sure that most of the draws from the proposal do not get rejected due to a acceptance probability equal to 0, I'm thinking of using some transformations.

For example, for the components which take values in $]0,1[$, I'm thinking of using a logit transformation. For $]0,+\infty[$ a $\log$ transformation. For $]-1,1[$, I will translate to $]-\pi/2,\pi/2[$ and apply $\tan$ .

What other transformations maybe used? Is this a good policy? What precautions should I take to choose good transformations?

An old man in the sea.
  • 5,070
  • 3
  • 23
  • 57
  • 2
    Make sure you do not forget the Jacobians. – Xi'an Dec 02 '18 at 16:52
  • @Xi'an the model I'm using makes it too hard to compute the jacobian. By hand, I simply do not know how to. By PC, I've tried defining the likelihood in Mathematica, then derive (1st derivative) the corresponding expression. with a sample size of 10, and my PC runs out of memory. Maybe, I'm missing something. To find the 1st derivative, I had to use some formulas involving derivation of a scalar function w.r.t. a matrix, and the kronecker product, and vec operator. How does one usually find the Jacobian? Sorry for this last question. ;) – An old man in the sea. Dec 02 '18 at 17:17
  • Do you plan on having non-zero off-diagonal elements in $\Sigma$? – Robin Ryder Dec 02 '18 at 17:18
  • @RobinRyder at the moment, I don't think so. However, I would like to. – An old man in the sea. Dec 02 '18 at 17:19
  • @Xi'an in my previous comment, I mixed jacobian with hessian. Sorry. – An old man in the sea. Dec 02 '18 at 17:40
  • Derivatives of $\log$, $\tan$ or logit are not particularly hard to find... – Xi'an Dec 02 '18 at 19:26
  • @Xi'an You're right. The location vector and scale matrix of the proposal depend on the gradient of the likelihood which involves some bothersome terms. I didn't notice you were talking about the 'jacobian' of the transformation, and not of the likelihood... – An old man in the sea. Dec 03 '18 at 10:32

1 Answers1

2

It is perfectly valid to reparametrize your model before implementing MCMC. Two caveats, as mentioned in the comments: (1) you need to calculate the Jacobian of the change of the variable; (2) depending on the problem, this can make it more difficult to think about the correlation between the parameters. Depending on the problem, it can be easier or harder to find a proposal that allows the chain to mix well.

However, since you are thinking of updating the components independently, another option is to forgo the $t$ distribution for certain parameters. In particular, your transition kernel can update only one parameter at each step, instead of updating all parameters at once. You can then choose a proposition which won't (at all/too often) propose values outside of the interval of support.

Edited to add: this answer by jbowman is very relevant and more detailed.

Robin Ryder
  • 1,787
  • 1
  • 11
  • 16
  • Just to be sure, the only place where I have to calculate the jacobian is for the prior, right? In the likelihood, nothing changes. – An old man in the sea. Dec 03 '18 at 19:39
  • @Anoldmaninthesea. You compute the jacobian for the change of variables only once, and you then write everything in terms of the new parameterization. – Robin Ryder Dec 03 '18 at 19:59
  • Robin, I have a doubt. If some of the parameters are reparametrized for the proposal, but the prior is given with respect to the reparametrization(not the original), do I still need to consider these parameters when computing the jacobian? – An old man in the sea. Feb 24 '19 at 16:57
  • @Anoldmaninthesea. Are you still using the original somewhere? Or are all of {prior, likelihood, proposal} defined with the reparametrization? – Robin Ryder Feb 25 '19 at 08:00
  • I'll be using the original only in the likelihood (but that's the same as using the reparametrization) – An old man in the sea. Feb 25 '19 at 15:08
  • 1
    In that case, if I understand correctly, you no longer need a Jacobian, since you can effectively express everything in terms of the new parameters. – Robin Ryder Mar 01 '19 at 12:50