I have an MCMC procedure that samples latent variables $h_1, \dots, h_T$. It is based on Shephard and Pitt (1997), https://doi.org/10.1093/biomet/84.3.653. Let $f$ be the true conditional posterior and $g$ a proposal density, and $h_{i:j}$ the current block for $t=i, \dots, j$. Then at MCMC iteration $r$ it consists of the following steps:
Randomly partition the sequence of $h_t$ over $1, ..., T$ into $K+1$ blocks.
Sample the latent variables of each block in turn, conditional on the remaining blocks:
2a. Find the parameters of a proposal density $g$ based on $h_{i:j}^{(r-1)}$
2b. Draw $h_{i:j}^*\sim g$
2c. Draw $u_1\sim U(0,1)$
i. If $u_1<f/g$, accept $h_{i:j}^*$ and proceed to Step 2d
ii. If $u_1>f/g$, return to 2b
2d. Draw $u_2\sim U(0,1)$, and let $$ r=\min\left\{1, \frac{f(h_{i:j}^*)\min[f(h_{i:j}^{(r-1)}), g(h_{i:j}^{(r-1)})]}{f(h_{i:j}^{(r-1)})\min[f(h_{i:j}^*), g(h_{i:j}^*)]}\right\} $$
i. If $u_2<r$, let $h_{i:j}^{(r)}=h_{i:j}^*$
ii. If $u_2>r$, let $h_{i:j}^{(r)}=h_{i:j}^{(r-1)}$
My problem is that my sampler occasionally cannot provide a good proposal using the proposal density, meaning that the accept-reject step rejects the proposals for a significant amount of attempts. This happens because the block is too big, but what is "too big" of course varies.
My question is: can I escape long spells of an unsuccessful accept-reject step without destroying the properties of the Markov chain (i.e., avoid screwing up the target)? I am thinking something along the lines of if $N$ proposals from the approximate model have been rejected, then go back to 2a, and perturb the starting point so that the approximate model is slightly different, or maybe split the block in two so that it is easier to find a proposal that is accepted.
