2

as far as I understand, AIC or BIC can be used to judge the goodness of fit between two GLMs, but they two has to be one nested inside the other. If I have two models that are not nested and based on different data, is there a way to compare the goodness of fit? TIA

Ting Ting
  • 21
  • 1
  • If they are based on different datasets then how will you distinguish between differences due to different models and differences due to different data? – mdewey May 19 '16 at 13:42
  • I agree with @mdewey It's hard to imagine a meaningful scenario where you would want to compare GOF of two models fit to different data. – Jacob Socolar May 19 '16 at 17:59
  • Maybe a duplicate: http://stats.stackexchange.com/questions/116935/comparing-non-nested-models-with-aic – kjetil b halvorsen Mar 04 '17 at 18:04

1 Answers1

2

One thing you can do is to get the residuals from each model and compare them. You could use a t-test but it's probably more useful to simply compare absolute values and then ask yourself if the improvement in fit is worth it. This is not a formal test but it's still useful.

Various plots of the outputs can be useful; I've found it helpful to look at Tukey mean difference plots, QQ plots of the different residuals; box plots of residuals and so on.

Peter Flom
  • 94,055
  • 35
  • 143
  • 276