What does the distribution of samples from an MCMC method converge to without repeated samples?

Question

Suppose I have an absolutely continuous distribution with density $f(x)$ and I use an mcmc sampler which has accept/reject step to sample from this distribution. In the final samples, there are some singular points (i.e., samples on top of each other). Does the distribution of the samples without these singular points converge to anything meaningful?

Answer 1

We addressed this problem in our 2011 vanilla Rao-Blackwellisation paper. The limiting distribution of the unique simulations in the Metropolis-Hastings sequence is associated with the density $$\tilde\pi(x)\propto\pi(x)\bar{\alpha}(x)\quad\text{where}\quad\bar{\alpha}(x)=\int_{\mathcal X}\alpha(x,y)q(y|x)\,\text{d}y$$if

• $\pi(\cdot)$ is the original target of the Metropolis-Hastings algorithm
• $\alpha(\cdot,\cdot)$ is the Metropolis-Hastings acceptance probability
• $q(\cdot|\cdot)$ is the Metropolis-Hastings proposal or kernel

(In the above excerpt from the paper, $\mathfrak z_i$ denotes the $i$th accepted value in the MCMC chain and $\mathfrak n_i$ the number of times it is repeated in the MCMC chain.)