2

Suppose I have parameters $\theta_0$ and $\theta_1$ with prior $$ p(\theta_0,\theta_1)=p(\theta_0|\theta_0<\theta_1)p(\theta_1),$$

that is, $\theta_0$ is less than $\theta_1$. The distributions are assumed to be continuous on their support. Apart from the likelihood, there are no other terms involving $\theta_0$ and $\theta_1$. I'm trying to consider potential sampling procedures for these parameters.

My current MCMC procedure is:

  1. Propose a value for $\theta_1$ using a random walk update.
  2. Use the independence sampler to sample a value of $\theta_0$ that is restricted to be less than the proposed value of $\theta_1$ using the appropriate truncated density (with the correction).
  3. Evaluate Metropolis-Hastings accept/reject.

Does anyone have ideas for other proposals here? The trick is that the support of $\theta_0$ changes, so a random-walk Metropolis step from the previous value of $\theta_0$ isn't guaranteed to fall in the support of $\theta_0$ given an updated value of $\theta_1$, so this set of draws would be rejected in a random-walk Metropolis scheme. This results in very few accepted proposals and increases the number of iterations needed.

My question is similar to a question posited here that was never answered.

Alex
  • 37
  • 5
  • 1
    This answer, to a slightly different question, may be useful: https://stats.stackexchange.com/questions/73885/mcmc-on-a-bounded-parameter-space/73897#73897 – jbowman Aug 14 '19 at 19:13
  • @jbowman: Thanks, I'll check it out! – Alex Aug 14 '19 at 21:47
  • You could transform the parameters to $\theta_1, \delta$ where $\delta > 0$ and $\theta_0 = \theta_1 - \delta$. Then the constraint disappears and the support is the same at each iteration. – jbowman Aug 15 '19 at 14:51

1 Answers1

2

If the question is about simulating from a target with a density of the form$$f(\theta_0,\theta_1)\mathbb I_{\theta_0<\theta_1}\tag{1}$$a Metropolis-Hastings algorithm does not need to enforce the constraint (1) in the proposal as proposed values that do no satisfy the constraint (1) will be rejected at the accept-reject step.

A natural approach in this case is to use a Metropolis-within-Gibbs approach that

  1. simulate $\theta_1$ conditional on $\theta_0$ and possibly the current value of $\theta_1$, $\theta_1^-$, from a proposal $p_1(\theta_1|\theta_0, \theta_1^-)$ and accept or reject through a Metropolis-Hastings step;

  2. simulate $\theta_0$ conditional on $\theta_1$ and possibly the current value of $\theta_0$, $\theta_0^-$, from a proposal $p_0(\theta_0|\theta_1, \theta_0^-)$ and accept or reject through a Metropolis-Hastings step

    This way, the constraint is taken into account symmetrically between $\theta_0$ and $\theta_1$.

Xi'an
  • 90,397
  • 9
  • 157
  • 575
  • Thanks for the response. Could you please clarify how this would work in the following scenario? The current value of $\theta_0$ is $\theta_0^-$. Suppose after proposing and accepting $\theta_1^*$ a new value for $\theta_1$, $\theta_1^* < \theta_0^-$. Upon proposing a new value $\theta_0^*$ for $\theta_0$, the acceptance ratio is $$\frac{\pi(\theta_0^*, \theta_1^*)}{ \pi(\theta_0^-, \theta_1^*)}\frac{f(\theta_0^- | \theta_0^*)}{f(\theta_0^* | \theta_0^-)},$$ but the value of $\pi(\theta_0^-, \theta_1^*)$ will be zero because $\theta_0^-$ does not fall in the support determined by $\theta_1^*$. – Alex Aug 15 '19 at 16:34
  • I just thought through it. $\theta_1^*$ would never had been accepted in the first place because it was less than the current value of $\theta_0$ when it was drawn. – Alex Aug 15 '19 at 18:35
  • Indeed, at each step, the constraint cannot but hold. – Xi'an Aug 16 '19 at 07:26
  • 1
    I didn't read the paper I posted carefully enough--@Xi'an is right, this is the correct approach. – Sheridan Grant Aug 16 '19 at 18:07