4

Let's say I have to find the posterior distribution of a Bayesian estimate. Why should I use an MCMC chain of length 10,000 but not do a Latin Hypercube Sampling of 10,000 samples and calculate* the corresponding posterior distribution values? What is the advantage of using MCMC over LHS?

*calculation: (prior) * (likelihood) / (normalizingConstant) at the lhs points.

Safwan
  • 255
  • 2
  • 11

1 Answers1

4

MCMC accomplishes posterior estimation via sampling, which is not to say that sampling is the point -- I think that the focus on sampling is why you proposed LHS as an alternative. But with MCMC, we're concerned with inference. The quintessential use case for MCMC is making inferences about the posterior when the Bayes rule computation is intractable, so I don't know how you plan on going about computing the density at LHS points. That is, sampling is a cheaper way to estimate posterior densities in cases where the posterior is not analytically soluble and numerical integration is too slow.

Even if the computation were possible, the return would just be a list of densities at LHS points -- I don't know how that helps to answer the question of what the posterior density looks like: what is its mode? What is is mean? Is it multi-modal? Left- or right-skewed?

Sycorax
  • 76,417
  • 20
  • 189
  • 313
  • Once we get the densities at LHS points, why can't we use interpolation for getting the density values at more points and plot the density function? It will give the information about the look, mode, skewness, etc. – Safwan Mar 29 '16 at 14:11
  • 1
    (1) It's not as simple as "getting the densities." That's explained in the first paragraph. (2) You would have to interpolate in many dimensions at once, and there's no reason to believe that a linear interpolation will have any resemblance to the posterior. – Sycorax Mar 29 '16 at 14:13
  • Regarding the computation, we have to find the product of prior and likelihood anyway for doing MCMC. The only difference is that we will have to get the normalizing constant in the case of manual computation of densities at LHS points. Is it the normalizing constant that makes the computation intractable? Is it possible to explain using an example? Thanks a lot in advance. – Safwan Mar 29 '16 at 15:20
  • @MusafirSafwan Try working out the posterior density for a hierarchical normal-normal model. – Sycorax Mar 29 '16 at 19:11