VIF scores for ordinal independent variables

Question

I suspected there was a high degree of multicollinearity in the independent variables of my data. Each of these variables is ordinal. The original model is

library(logistf)
EC_all <- logistf(Erad_contr ~ Entry_risk + Entry_conf + Establishment_risk+ Establishment_conf + Spread_risk + Spread_conf+ Impacts_Risk + Impacts_Conf, data = Published, family = "binomial")

I then attempted to get VIF scores using the following:

library(car)
EC_test <- lm(Erad_contr ~ Entry_risk + Entry_conf + Establishment_risk+ Establishment_conf + Spread_risk + Spread_conf+ Impacts_Risk + Impacts_Conf, data = Published)
vif(EC_test)

                         GVIF Df GVIF^(1/(2*Df))
Entry_risk          7.882987  3        1.410745
Entry_conf         14.858967  3        1.567947
Establishment_risk  8.755895  3        1.435655
Establishment_conf 26.363955  3        1.725183
Spread_risk         7.105005  4        1.277749
Spread_conf         8.517452  3        1.429064
Impacts_Risk        7.951980  4        1.295864
Impacts_Conf        9.266215  3        1.449274

Should I be looking at GVIF which seems very high, or GVIF^(1/(2*Df)) which seems more normal. Regardless, have I done this correctly? I did not create dummy variables to do this, and have read that you should do this for categorical data, but I have not found much information on using ordinal data. If this is incorrect, how should I calculate VIF scores, or is there a better alternative?

UPDATE

Please note this is for a slightly different model shown below. But the point is the same. The original model is:

EC_Conc <- glm(Erad_contr ~ Conc_Risk+Conc_Conf, data = Published, family = "binomial")

I have attempted to create dummy variables as such:

For_Vif <- fastDummies::dummy_cols(For_Vif,select_columns = c("Conc_Risk", "Conc_Conf") )

and then created a model using each of the dummy variables as my independent variables and attempted to get VIF values:

VifModel3 <- lm(Erad_contr ~ Conc_Risk_Vlow+Conc_Risk_Low+
Conc_Risk_Med+Conc_Risk_High+Conc_Risk_Vhigh+ +Conc_Conf_Low+Conc_Conf_Med+Conc_Conf_High+Conc_Conf_Vhigh, data = For_Vif)

vif(VifModel3)

This yields the error

Error in vif.default(VifModel3) : 
  there are aliased coefficients in the model

Is this closer to correct rather than what was done before? How can I fix this error message and get my VIF scores?

UPDATE 2

As suggested by @Randcelot, I removed the lowest category for each of the variables in the lm.

VifModel3 <- lm(Erad_contr ~ Conc_Risk_Low+Conc_Risk_Med+Conc_Risk_High
                +Conc_Risk_Vhigh+Conc_Conf_Med+Conc_Conf_High+Conc_Conf_Vhigh, data = For_Vif)
vif(VifModel3)

                  Conc_Risk_Low       Conc_Risk_Med           Conc_Risk_High 
                12.951637                 21.451194                 20.794598 
                Conc_Risk_Vhigh    Conc_Conf_Med        Conc_Conf_High 
                 1.976190                  4.152511                  4.469138 
                Conc_Conf_Very_high 
                 1.532027

There are multiple VIF scores for each variable. Conc_conf looks acceptable for each. Whereas for Conc_risk vhigh looks acceptable while the others do not. Is it safe to assume that since some of the scores are very high, there is multicollinearity here? Seeing as there are only two variables here, I guess I can remove either of the independent variables?

score 1 · Accepted Answer · answered Jul 18 '20 at 06:39

1

The error you have now has to do with multicollinearity. When you made dummy variables for Conc_Risk and Conc_Conf, you made one for every possible value of each variable. There are five categories, and you made five dummies. However, you can only include four in the model.

The problem arises because any four of the dummy variables are always perfectly collinear with the fifth (you can always tell the value of the fifth dummy variable from the other four as it will be zero if any of the other four are one and it will be one otherwise). When you include all five, this leads to the multicollinearity error you got, since one of the five dummy variables is basically an alias for the information held in the other four. If you take out one of the dummy variables though, the error will go away and you won't lose any information (since you can tell what the value of the fifth dummy is from the other four).

It's traditional to leave out the lowest valued dummy variable, so that all of the other coefficients can be interpreted as the change associated with the increase from a base value, but you can choose any dummy variable to take out.

For more on the error check out this related post: What are 'aliased coefficients'?

answered Jul 18 '20 at 06:39

Randcelot

71
4

1

Thanks very much for this reply. I have done what you said, it seems to have worked perfectly. I have added this in a 2nd update in the original question. Are my assumptions I've stated at the end of the post correct? – Harry Jul 18 '20 at 10:08
1

I'm also wondering if there will also be collinearity between dummy variables of the same variable. Surely this will be reflected in the VIF scores? For example, if I calculate VIF when only one dummy variable of a variable is present, the VIF scores drop considerably. – Harry Jul 18 '20 at 10:33
Okay, yes, I believe you're assumptions at the end are correct. And also yes you may not need to remove either of the two independent variables entirely, you're issue may be solved by removing a few of the dummy variables. I would try removing each of the high vif dummy variables one at a time and seeing how much the other vif's go down. You should definitely be able to keep at least `conc_risk_very_high` – Randcelot Jul 18 '20 at 14:11
1

There definitely could be multicollinearity between dummies of the same variable, but I don't think there has to be. For example if one of your 4 dummy variables was always 0, then you might as well not include it as a category at all, because there are no observations in the category, but also because the other 3 dummy variables combined could tell you the value of the 4th. The same *I think* could be true if the dummy variables is true only a very small percentage of the time (the other three combined could reasonably predict it's zero?). Honestly I'm not totally sure – Randcelot Jul 18 '20 at 14:17
1

You can also check it this link which has a good answer for interpreting gvif going back to your very original problem! It also has a super complicated answer about ellipsoids I don't understand but if you're into that it's there https://stats.stackexchange.com/questions/70679/which-variance-inflation-factor-should-i-be-using-textgvif-or-textgvif – Randcelot Jul 18 '20 at 14:17

VIF scores for ordinal independent variables

1 Answers1