0

Say I have some data $X \in \mathbb{R}^{M\times N}$, such that $X_i \sim Normal(\mu, \Sigma)$. It's easy to compute the likelihood of $\Sigma$ conditioned on $X$ and $\mu$: $$ L(\Sigma)=p(X|\mu, \Sigma) = \prod_i normpdf(X_i;\mu,\Sigma) $$ Now I want to draw samples of $\Sigma$ under this likelihood. This is (or at least, seems to me) computationally difficult using MCMC methods, given the large $M$ that I am dealing with (on the order of 2000). So I was wondering: could I do this instead by bootstrapping? That is, would it make sense to create (many) samples $X^*$ by drawing with replacement from the columns of $X$, and computing the sample covariance of each $X^*$? Would the resulting samples $\Sigma^*$ approximate $L(\Sigma)$?

I've done some simulations with small $M$ where I can do Metropolis Hastings, and the distributions I get from MH in that case are very similar to those I get by bootstrapping, so I'm optimistic, and intuitively it seems right, but I'd like to have a more formal understanding/justification.

Edit: I guess another way to phrase my question is: is the sampling distribution of the MLE of $\Sigma$ identical to the likelihood over $\Sigma$ given $X$?

Ruben van Bergen
  • 6,511
  • 1
  • 20
  • 38
  • Basically you are just describing a parametric bootstrap. I've flagged it as duplicate because your confusion seems to stem from this distinction, understanding that both are valid, and using the appropriate terminology there. – AdamO Nov 30 '16 at 13:49
  • I'm not sure that's the root of my question. I understand that I'm describing a non-parametric bootstrap. As I tried to clarify in my edit, though, my question really pertains to the relationship between the sampling distribution of an estimator (in this case, the MLE of the covariance), and the likelihood over the estimated variable (the covariance) conditioned on a fixed data sample. I do see that this may not be clear from my original question though. Do you think I should formulate new question? Or would you say this question is already answered as well? – Ruben van Bergen Dec 02 '16 at 10:02

0 Answers0