I am implementing a Gibbs sampler in order to estimate the parameters of a mixture model.
Assuming that the parameters are contained in a vector $\theta$ what I will do is:
- Implement and run the sampler.
- Wait to reach a stationary distribution (so to remove the impact of my initialization); say that we reach the stationary distribution at iteration $n$.
- After iteration $n$, I start collecting the sampled parameters into a list $[\theta_n,...\theta_{n+m}]$
- Last, I evaluate the expected value of $\theta$ doing $$ \theta_{m,estimated} = \sum_{i=1}^{m} \theta_{n+i}/m $$
Now, I have some questions:
- Two subsequent samples $\theta_{n+i}, \theta_{n+i+1} $ will obviosly be correlated. Should I skip some samples in order to ensure independence between the samples I use to calculate my estimator? If yes, there is some standard heuristic to do this?
- Can I say that (if the sampler is properly implemented) $\theta_{m,estimated}$ is a consistent estimator of $\theta$?
