0

Since MCMC converges to target only after taking very large number of steps, what if I want to have just say 10 samples from target distribution? Do I still have to generate lots of samples, and then discard the rest?

Or like machine learning models, is there seperate training and testing part, where a model is trained to project an easy-to-sample distribution to our desired one?

Shihab Shahriar Khan
  • 125
  • 5
  • please let me know if this is duplicate. I tried searching around but don't really have the stat. vocabulary to do that properly – Shihab Shahriar Khan Dec 26 '18 at 07:18
  • Can you increase the number of steps? – user2974951 Dec 26 '18 at 08:15
  • sorry @user2974951, couldn't understand your question. Increase what steps? – Shihab Shahriar Khan Dec 26 '18 at 10:48
  • 2
    MCMC is used to sample from an unknown distribution, the more observations you can get the better as it better approximates it. Why do you want to select only 10 observations? Are you trying to randomly sample 10 observations from this distribution? If so, then you would pick 10 observations at random from all the available ones, however keep in mind that the observations are likely correlated. – user2974951 Dec 26 '18 at 11:07
  • I'm coming from ML. using GAN for example, you could ask for any number of samples, which will respect the distribution the data GAN was trained on. With MCMC now, we have to take that infinite walk every time we need to sample from target distribution, right? I mean is this how MCMC usually implemented in softwares? I'm sensing I'm missing something obvious here :( – Shihab Shahriar Khan Dec 26 '18 at 11:45
  • 2
    Yes, usually MCMC has a "warm-up" period during which you do not sample. This step is used to make sure that there are no problems at the beginning with convergence. If, however, you had strong knowledge / you knew that your MCMC would start at the right point of the distribution then you would have no need for a warm-up period. if so, you could ignore the warm-up and go straight to sampling. – user2974951 Dec 26 '18 at 12:29

1 Answers1

1

MCMC stands for "Markov chain Monte Carlo". There are two aspects to how long you will run your MCMC sampler: one for the Markov chain and one for the Monte Carlo part.

  1. Markov chain : Like user2974951 wrote in their comment, usually there is a burn-in or "warm-up" period, where you can't necessarily give your Markov chain a good starting value, so your chain takes time to get to areas of high probability. This part may be ignored and thrown away, since the samples are not representative of the distribution. However, if you have a strong knowledge of the areas of high probability, you can start your Markov chain from a point in that area, in which case a "warm-up" is not essential. In any case, this part does not factor into the sampling.

  2. Monte Carlo: To answer your question, I would need to know more about what you're using your samples for. But generally, if you're doing MCMC, samples are used for Monte Carlo averages. That is, if your target distribution is $\pi$, and you're using MCMC to estimate $E_{\pi}g = \int g(x) \pi(dx)$, then from a Markov chain sample $X_1, \dots, X_n \sim \pi$, the Monte Carlo estimator of $E_{\pi}g$ is $$\dfrac{1}{n} \sum_{t=1}^{n} g(X_t)\,. $$ That is a true average is estimated by a sample average. Now the question is, if you're given a distribution, and given samples from that distribution, then are 10 samples enough to estimate the mean of the distribution relatively well? The answer is almost unequivocally, no. 10 samples from the target distribution is nearly not enough, let alone the fact that MCMC gives you "corrupt" samples, since they are correlated. To understand how many samples after "burn-in" you may need, read this post and my answer there.

Greenparker
  • 14,131
  • 3
  • 36
  • 80
  • Thank you. So if I don't really care about population statistics, but only on generating samples, I don't really need the Monte carlo part, right? Bishop's PRML mentions taking every Mth sample if we want them de-correlated, so there is no avoiding generating lots of samples anyway... – Shihab Shahriar Khan Dec 26 '18 at 14:54
  • 1
    @ShihabShahriar Why are you generating samples if you aren't interested in population statistics? I am curious. Also, why are you interested in removing correlations by taking the Mth sample, when you aren't interested in population statistics? – Greenparker Dec 26 '18 at 23:56
  • 1
    My understanding is, this is what Generative Adversarial Network does. You've got dataset of 10,000 pics of cute cats to train a model, and want to build an app that generate new cat pics whenever user asks for, by sampling from the same unknown distribution that generated your dataset. I understand there are better alternatives to MCMC for this particular task, but this is an example where population params aren't really our goal. – Shihab Shahriar Khan Dec 27 '18 at 04:52
  • @ShihabShahriar Ah ok. That's helpful. Yeah, so then you also don't need to worry about thinning ("taking every Mth draw"). But you do need to be careful about burn-in. – Greenparker Dec 27 '18 at 07:07