Why is Bayesian data analysis done if we already know the distribution of the parameters?

Question

I am trying to learn Bayesian data analysis, so what I see is that most computations are carried out using MCMC simulations.

So far as I understand, for simulating MCMC we need to know the distribution up to a constant (i.e., we need to know parameters) and then we can compute stuff related to it using the samples generated. But if we know distribution then why would we go about doing analysis? What am I missing here?

Including a small example in your explanation or any reference that can explain this with toy examples will be helpful.

Answer 1

The fundamental answer is that even with a function that gives the distribution (or the distribution up to a constant), we often still can't calculate analytically the quantities we are interested in e.g. the mean, the median, credible intervals. So instead we need some samples to get an approximate answer for whatever we're interested in.

Suppose you're interested in some parameter $\theta$ which can take real values and you have some function $\pi$ which tells you the density of your desired distribution for any value of $\theta$ (in a Bayesian setting this is usually the posterior distribution, but really this holds for any distribution of interest).

As you've mentioned, we usually want to know more than just this information. For example, we probably want to know the mean of this distribution or some credible intervals. To calculate the mean we need to calculate $$\int_\mathbb{R} x\pi(x) \mathrm{d}x$$ Sometimes this is an integral we can do but a lot of the time it's not. So we'll have to do it approximate it somehow and the easiest way (especially true in high-dimensional cases) is by getting samples from the distribution and taking the average.