4

As far as I know, the variance inflation factor is not computed with pseudo-$R^{2}$ or generalized $R^{2}$ in binary outcome models (e.g. logistic regression).

Are there other measures of multi-colinearity than VIF appropriate to such models?

Why ought or ought not we think about multi-colinearity in such models?

Alexis
  • 26,219
  • 5
  • 78
  • 131
  • You can have a look at the discussion here: https://www.researchgate.net/post/How_to_test_multicollinearity_in_logistic_regression – kjetil b halvorsen Aug 23 '14 at 12:29
  • @kjetilbhalvorsen did you mean to post the same link in your second comment? The first comment briefly stated that one might construct a VIF using McFadden's pseudo-$R^{2}$, but there's not really any discussion about why that is or is not important to do this. For example, introductory textbooks on regression will stress the importance of checking for collinearity (e.g. by using VIF) in a multiple linear regression context, but ignore the issue when presenting logistic regression. Trying to understand why. – Alexis Aug 23 '14 at 17:09
  • There is relevant discussion here: (section 3.3) http://www.ats.ucla.edu/stat/stata/webbooks/logistic/chapter3/statalog3.htm – kjetil b halvorsen Aug 23 '14 at 17:42
  • googling for "variance inflation factor for logistic regression" gives other relevant hits. Try that, and come back if you cannot solve your problem that way. Multicollinearity is problematic with logistic regression the same way it is with linear regression, so iy should be possible to transfer some techniques, but I dont know what is best! – kjetil b halvorsen Aug 23 '14 at 17:44

1 Answers1

4

There can be VIFs for generalized linear models like logistic regressions. They just shouldn't be calculated the way that the Wikipedia page currently shows, based on $R^2$ for regression of each predictor against all the others. Consider the VIF definition provided by that page:

the quotient of the variance in a model with multiple terms by the variance of a model with one term alone.

The information needed to calculate the VIF is in the variance-covariance matrix of the coefficient estimates. The derivation on that Wikipedia page starts from the equation for the variance-covariance matrix of an ordinary least squares (OLS) model. That then leads to the usual equation for VIF.

With a generalized linear model there is still a coefficient variance-covariance matrix, but it comes from maximum-likelihood fitting rather than a specific analytic form. So the VIF calculation used for OLS wouldn't be appropriate.

The vif() function in the R car package calculates generalized variance inflation factors

interpretable as the inflation in size of the confidence ellipse or ellipsoid for the coefficients of the term in comparison with what would be obtained for orthogonal data.

For an unweighted OLS model this calculation should be equivalent to the usual equation while providing an important generalization for other models. You can inspect the code by loading the car package and then typing getAnywhere(vif.default) at the R command prompt.

EdM
  • 57,766
  • 7
  • 66
  • 187