1

Imagine I have the following 7 glmm models where $b_1$ through $b_3$ are fixed effects.

$M_1 = y \sim b_1 \times b_2 \times b_3$

$M_2 = y \sim b_2 \times b_3$

$M_3 = y \sim b_1 \times b_3$

$M_4 = y \sim b_1 \times b_2$

$M_5 = y \sim b_3$

$M_6 = y \sim b_2$

$M_7 = y \sim b_1$

DIC can be used to asses the relative fit of models. However, I would like to know what comparisons would be valid to make because I am unsure and struggling to know what route to take. Imagine the following scenario, where the DIC for each model is shown in red.

enter image description here

I am not sure what would and would not be valid comparisons/correct interpretations of using DIC compare the models. Would it be valid to compare, for example, the following:

A) $M_1$ with $M_2$ - I think this is a valid comparison, and shows whether dropping the parameter $b_1$ improves the fit of the model. If $\Delta$ DIC > 2 then model fit is improved by dropping the term.

B) $M_1$ with $M_5$ - I suspect that this is not a valid comparison. It tests whether dropping two parameters at the same time improves model fit. I think I should first try dropping both $b_1$ and $b_2$ individually, then proceeding to testing $M_2$ and $M_3$ against $M_5$, retaining the most complex model where dropping terms does not improve fit.

C) $M_5$ with $M_6$ - I suspect that this is not a valid comparison. It shows the effect of substituting the parameters, which may mean different data is included.

Essentially it boils down to this. Can one only use DIC to compare models with sequentially dropped effects, or can I drop multiple effects simultaneously and/or compare models with different fixed effects?

Then how would I draw conclusions from this? Would it be best to present the model $M_1$ as the best fitting model, because it is the most complex model of all of my models where $\Delta$ DIC is <2 when dropping parameter $b_3$... or, that models $M_5$ and $M_6$ (and $M_1$) are the best fitting models when using parameters $b_3$, $b_2$, (and $b_1$) respectively.

rg255
  • 752
  • 2
  • 8
  • 27

1 Answers1

2

You can use DIC to compare any of the models to each other without considerations to nesting of effects as long as at least one of the below holds

  • The random effects structure is the same for all models
  • If the random effects structure is different then all models are fit using maximum likelihood

As far as the delta_DIC < 2 policy, I know burnham and Anderson suggested a deltaAIC cut-off < 2... but I am unsure how this will translate to DIC. But otherwise, IMHO you can just choose the best fitting model from all models

ashokragavendran
  • 409
  • 2
  • 6
  • so then I could look at all of my models, and just pick the one with the lowest DIC as being the best fit - in my example it would be model $m_5$ – rg255 May 31 '16 at 13:11
  • yes that is correct. That is the goal of using an IC based model selection strategy is that non-nested models can be compared. When you use an likelihood Ratio approach you can only compare nested models – ashokragavendran May 31 '16 at 13:14
  • and so I don't need to work from most complex (m1) -> least complex (e.g. m5) models and take the most complex model where DIC isn't substantially reduced? – rg255 May 31 '16 at 13:17
  • The point of the model selection is that you will have to ensure that you have explored the entire set of model space. In your example it is simple, but if you add a few more variables the # of models to fit increase substantially. You don't have to fit models in any particular order, just have to fit all possible models :) and then compare with your favorite IC criteria – ashokragavendran May 31 '16 at 13:23