0

Takeuchi’s information criterion is said to be the generalization of AIC to misspecified models. That publication presents DEGREES OF FREEDOM FOR NONLINEAR LEAST SQUARES ESTIMATION. From that source: enter image description here

This appears similar to a Tikhonov regularization. Would someone please explain what Takeuchi’s information criterion is in simple language? It would appear that calulating degrees of freedom is not simple. In addition to df calculations, are there other uses for TIC? If, as George Box says "Essentially, all models are wrong...." which application of AIC is not to misspecified models? Should we all be using TIC as opposed to AIC?

Carl
  • 11,532
  • 7
  • 45
  • 102
  • 1
    Note that when the model's not mis-specified (following Hansen & Sokel's notation), $\mathbb{J}=\mathbb{G}$ is the Fisher information matrix & $\operatorname{tr}\left(\mathbb{J}^{-1}\mathbb{G}\right) = p$. So your last question boils down to whether it's better either to approximate $\operatorname{tr}\left(\mathbb{J}^{-1}\mathbb{G}\right)$ under the assumption that the model isn't *badly* mis-specified (AIC) or to estimate it from the data (TIC). (Which will depend on what you're using the information criteria for, & on the amount of data you have.) – Scortchi - Reinstate Monica Feb 24 '21 at 12:33
  • @Scortchi-ReinstateMonica Thanks for the information. My limited understanding is that TIC appears to be regularization, and that may be useful when the misspecification is in part from inappropriately using AIC for an ill-posed problem, for example for censored data. – Carl Feb 24 '21 at 12:57
  • Yes: you can think of model selection with either AIC or TIC as maximizing a penalized likelihood over a set of models. Is there some particular class of models for censored data you've got in mind? - the bog-standard maximum-likelihood approach for non-informative censoring doesn't preclude use of AIC. – Scortchi - Reinstate Monica Feb 24 '21 at 15:11
  • @Scortchi-ReinstateMonica Indeed there is. For drug concentrations in blood as a function of time it is usually not possible to collect data to extinction for several reasons. Consequently, ML regression would not be appropriate, not because concentration curves are not random variables, and of course they are not, but because the data does not span the concentration space (i.e., space=time in that context). It is however amenable to [Tikhonov regularization](https://link.springer.com/article/10.1007/s10928-010-9167-z). – Carl Feb 25 '21 at 01:32

0 Answers0