2

I have 3 variables R&D Spend, Administration and Marketing spends. I wanted to calculate VIF and eliminate a variable for better fit to the model.

I tried to use the solution at https://stats.stackexchange.com/questions/155028/how-to-systematically-remove-collinear-variables-in-python

[8.3845707545599613, 4.0264055178945535, 7.5939835926809236]
dropping 'R&D Spend' at index: 0
[3.4365296868536528, 3.4365296868536528]
Remaining variables:
Index([u'Administration', u'Marketing Spend'], dtype='object')

on the same data

Gretl Output:

Variance Inflation Factors
Minimum possible value = 1.0
Values > 10.0 may indicate a collinearity problem

RDSpend    2.469
MarketingSpend    2.327
Administration    1.175
VIF(j) = 1/(1 - R(j)^2), where R(j) is the multiple correlation coefficient
between variable j and the other independent variables
Richard Hardy
  • 54,375
  • 10
  • 95
  • 219
Saqib Mujtaba
  • 121
  • 5
  • statsmodels vif uses the raw data and does not do any scaling or standardizing. It was written as a diagnostic tool for a given design matrix. Gretl might standardize the variables. – Josef Aug 18 '17 at 14:03
  • Thanks but do you know any link where reference to scaling is done? Also, if true which kind of scaling. – Saqib Mujtaba Aug 21 '17 at 07:07
  • I don't know Gretl except for having run a few examples. You need to check their documentation, or try to replicate their results using different scaling or standardization. – Josef Aug 21 '17 at 14:23

1 Answers1

2

Gretl's documentation for this is already given in the output. There is no standardization or anything (I just cross-checked by running the auxiliary regression manually). Gretl uses the standard meaning of "correlation coefficient", that is where an intercept is included.

In contrast, from the statsmodels code on http://www.statsmodels.org/dev/_modules/statsmodels/stats/outliers_influence.html#variance_inflation_factor, the function in turn calls statsmodels.regression.linear_model.OLS, and there in the docstring you can see the remark: "An intercept is not included by default and should be added by the user." Accordingly the explanation of the input in variance_inflation_factor is: "design matrix with all explanatory variables" (emphasis added).

(Admittedly, there may be rare cases where in the original model you want to omit the constant term but then you still want to include it for the VIF calculation, so the statsmodels implementation is perhaps not optimal.)

So could it be that you haven't explicitly included a constant term?

Sven S.
  • 524
  • 2
  • 8