3

My model's chi square says my 'full' model isn't significantly better than the null model. But my -2LL is 80 (isn't this pretty low?) and my Nagelkerke $R^2$ is .20.

On the other hand, there isn't a single significant IV in my model... So how should I interpret the lower -2LL and the not that small .20 Nagelkerke $R^2$?

gung - Reinstate Monica
  • 132,789
  • 81
  • 357
  • 650

1 Answers1

4

I do not think the -2LL is especially interpretable on its own. The $\chi^2$ test between the null model (typically intercept only model) and your full model (with all IVs) is: $$\chi^{2}_{df_{full} - df_{null}} = (-2LL_{null} - -2LL_{full})$$

The fact that the $\chi^2$ test is not significant suggests that on average, your predictors are not doing better than chance, which is entirely congruent with none of your IVs being statisticall significant. Another excellent interpretation (thanks to @gung) is that the $\chi^2$ tests that all parameter estimates are simultaneously equal to zero. If that is true for most (but not one or two) the overall, multidegree of freedom test will be non significant.

Suppose we take the not small Nagelkerke $R^2$ as meaningful. What are the possibilities? Perhaps 1-2 of your IVs are excellent predictors, however, they are highly correlated with each other. Consequently, neither are strong unique predictors, and are not statistically significant. If you also have 10 other predictors in the model with near null effects, 12 predictors 1-2 of which have some effect, would not necessarily be sufficient to make the overal model appear to be doing better than chance, although perhaps the 1-2 are sufficient to improve prediction and the Nagelkerke $R^2$.

What could you try? Examine a correlation matrix of your IVs. Are any of them highly correlated? Compare a series of one predictor models to see if a few of your predictors have effects when they are by themselves.

Another, somewhat controversial, option is stepwise regression. Stepwise regression uses an algorithm to search for significant predictors. There are several variants (forwards, backwards, and stepwise). In this instance, I would think that backwards would be optimal. This will drop predictors one at a time until all remaining predictors are statisticall significant. This invalidates any hypothesis testing, because it selects variables based on their significance. That is, whatever is left in the model is by definition significant, so the statistical tests of each parameter are no longer a valid way to verify some a priori hypothesis. However, it can be useful for learning and exploring your data. The results of a stepwise regression combined with theory, could form the basis of a new study where you collect new data, and test a priori specified hypotheses.

How sensible this is depends a lot on your specific situation. If you only have a handful of predictors to start with, there is little point. However, there are many legitimate cases now where people may have dozens or hundreds of predictors. This could be onerous to go through by hand to find out if any particular subset of them are leading to the high value of Nagelkerke's $R^2$.

A final caveat, that all the techniques I suggested involve a degree of data exploration (to outright data mining). Some people feel strongly that these cause more harm than good. So long as they are used in a true data mining context or as a supplement (not replacement) to theory, I think they can be useful. When used to supplement theory, I view them as hypothesis generating, not hypothesis confirming. The confirmation comes when you collect new data or use a different data set that was not explored to verify results from your exploration.

Joshua
  • 1,913
  • 15
  • 16
  • @gung Thanks as always for the thoughts. I have edited in some of your suggestions. I also added stronger caveats regarding stepwise selection, although I left it in for the reasons stated in the answer. However, I would be happy to hear your perspective on why it may be better to not mention it at all and would encourage you to write an answer detailing :) I am certainly not a huge fan of the technique. – Joshua Jul 23 '12 at 15:14
  • 1
    Can't thank you enough for that well explained answer ! – studenteconomics Jul 23 '12 at 18:08
  • @studenteconomics, if Joshua's answer provided the information you needed, you may want to accept it by clicking the check mark to the left of the answer. – gung - Reinstate Monica Aug 09 '12 at 02:38