3

I just want to make sure my understanding of how we estimate DIC using MCMC output is correct, as the wikipedia page is somewhat confusing.

Defining DIC by

$$DIC = p_D + \bar{D}$$

with

$$D(\theta) = -2 \log p(y|\theta) + C$$

$$p_D = \bar{D} - D(\bar{\theta})$$

I have a few questions about this:

Firstly, as far as I can tell, the Wikipedia page never actually explicitly defines $\bar{D}$, but I am guessing that it is the expected value of $D$ wrt $\theta$? ie

$$\bar{D} = E_\theta [D(\theta)]$$

Is this correct?

Secondly, $\bar{\theta}$ is defined as the "expectation of $\theta$". Does this mean we take $\bar{\theta}$ as our posterior estimate of $\theta$, the posterior sample mean? As that would be our best estimate of the posterior expectation of $\theta$.

Thirdly, if indeed $\bar{D} = E_\theta [D(\theta)]$, then given our posterior sample

$$\theta_1, \theta_2, \dots, \theta_n$$

Is $\bar{D}$ simply estimated by calculating $D(\theta_i)$ for each $\theta_i$ in our posterior sample, and then averaging them?

Thanks in advance. I'm still new to Bayesian so even these simple ideas confuse me.

Xiaomi
  • 2,276
  • 6
  • 19
  • $\bar\theta$ can be any point estimator deduced from the posterior. This indeterminacy is one of the many shortcomings of DIC, which should not be used imho. – Xi'an Nov 09 '18 at 10:41

1 Answers1

2

The short answer is that you are right. The clearest explanation that I know is in "Statistical Rethinking" by Richard McElreath. He puts it like this:

So define $D$ now as the posterior distribution of deviance,. This means we compute deviance...for each set of sampled parameter values in the posterior distribution. So if we draw 10,000 samples from the posterior, we compute 10,000 deviance values. Let $\bar{D}$ indicate the average of $D$. Also define $\hat{D}$ as the deviance calculated at the posterior mean. This means we compute the average of each parameter in the posterior distribution. Then we plug those averages into the deviance formula to get $\hat{D}$ out...DIC is calculated as:$$\bar{D}+(\bar{D}-\hat{D}) = \bar{D}+p_D$$

I hope that helps.

JeremyC
  • 291
  • 1
  • 5
  • This is correct for Spiegelhalter's definition of DIC. However, there are at least 8 definitions of DIC, and the answer depends on which definition. I have provided 3 examples of definitions on another question, which may be of interest to some: https://stats.stackexchange.com/questions/331928/dic-waic-in-jags/476780#476780 – Earlien Jul 12 '20 at 05:09