1

I have a model I want to fit parameters to. I know the likelihood $P(D|\theta)$. Let's say I have two parameters with prior belief, $\theta^1 \sim \mathcal{N}(\mu_2,\sigma_2^2)$, $\theta^2 \sim \mathcal{N}(\mu_1,\sigma_1^2)$. However, I know that both parameters are positive and so I bound it below by 0.

For the proposal distribution $q$, I'm using a normal centred at the previous value of $\theta^i$, with variance some scalar multiple of the initial value of the parameter (so different for each parameter). I can draw a random smaple from this proposal distribution to obtain a new sample, \theta'. (I can then repeat this for each parameter using their respective proposal distribution). However, I'm keeping in mind that the parameterds are bounded below by 0.

To calculate the acceptance ratio, I need to find

$\alpha = \min\{1,\frac{\mathcal{L}(D|\theta')q(\theta_{t-1}|\theta')P(\theta')}{\mathcal{L}(D|\theta_{t-1})q(\theta'|\theta_{t-1})P(\theta_{t-1})}\}$.

Normally, I'd be happy that I could ignore the proposal distribution here as it is symmetric. However, is this still this case given that I don't want negative parameter values? I'm a bit confused as to how I calculate $P(\theta_i)$. Does it just mean calculate $\frac{1}{\sqrt{2\pi \sigma_i^2}}e^{-\frac{(\theta^i - \mu_i)^2}{2\sigma_i^2}}$, and plug that number in? (So the value of the pdf, rather than the 'probability')? How do I incorporate the 'truncatedness' of the distribution when doing this? Or would I be better off using a gamma prior?

Example:

Would the following be correct: Let's just consider one parameter $\theta$ with prior belief $\theta \sim \mathcal{N^+}(0.16,0.06^2)$. Let's say $\theta_{t-1} = 0.14$. I draw a new proposal from the proposal distribution $\mathcal{N^{+}}(0.14, 0.05^2)$, which gives me $\theta' = 0.15$, say. Then the exponential part of the proposal pdf will be the same for both the numerator and denominator of the acceptance ratio, so I only need to deal with the normalising constant. So the proposal ratio part of alpha just becomes $\frac{\Phi(\theta_{t-1}/\sigma)}{\Phi(\theta' /\sigma)} = \frac{\Phi(0.14/0.05)}{\Phi(0.15/0.05)}$

Does this then mean that overall, my acceptance ratio becomes

$\alpha = \min\left\{1,\frac{\mathcal{L}(D|\theta')\Phi(0.14/0.05)e^{-\frac{(\theta'-0.16)^2}{2(0.06)^2}}}{\mathcal{L}(D|\theta_{t-1})\Phi(0.15/0.05)e^{-\frac{(\theta_{t-1}-0.16)^2}{2(0.06)^2}}}\right\}$.

user112495
  • 297
  • 1
  • 9
  • Could you clarify the "keeping in mind that the parameterds are bounded below by 0"? If you use a _truncated_ Normal as a proposal in MCMC it is no longer symmetric and you need to compute its normalising constant in the Metropolis acceptance probability. – Xi'an May 07 '20 at 12:03
  • @Xi'an I've added an example in my post. Is that correct, or have I misunderstood? – user112495 May 07 '20 at 18:03
  • You could also work with log-transformed parameters. – Björn May 07 '20 at 18:12
  • @Xi'an I thought I had? I've included the rest of the normalising constants (at least, what I thought they would be), but don't they cancel out, as the bits in the square roots only include the standard deviation which will cancel out from top to bottom? – user112495 May 08 '20 at 08:50
  • @Xi'an, Also, I'm referring to the second (larger) acceptance ratio in my post (not the general one higher up). Thanks! – user112495 May 08 '20 at 09:50
  • @Xi'an I've just worked through it again, and think I understand what you mean now. Though I think my ratio of $\Phi(...)$ is the other way round to yours. Should the ratio of normalisation constants be the other way round? – user112495 May 08 '20 at 10:48
  • 1
    yes $\alpha$ is now correct. – Xi'an May 08 '20 at 12:16
  • @Xi'an I'm now using adaptive MCMC with a truncated multivariate normal proposal. Calculating these normalisation constants is very expensive, and so amlooking for an alternative approach. Have you got any references on some kind of adaptive MCMC when all parameters are positive? – user112495 May 14 '20 at 14:58

0 Answers0