4

I am fitting a binomial logistic regression in R using glm. By chance, I have found out that if I change the order of my predictor variables, glm fails to estimate the model. The message I get is unexpected result from lpSolveAPI for primal test.

I am using the safeBinaryRegression package, so I am confident there are no separation issues between my outcome and predictor variables. However, I am not so confident that there are no quasi-separation issues among my predictor variables. Am I correct that if this is the case, then I might be running into multicolinearity, and this is the source of glm not being able to fit the model?

If so, my question is for advice on how to approach the issue. Should I look for the predictor variables highly correlated and omit one of them? Is there any convenient way of doing so for 11 categorical predictors?

What I see right now: 

lModel <- glm(mob_change ~ education + gender + start_age + income + dist_change + lu_change + dou_change + marriage + student2work + wh_change,
              data = regression_data, 
              family = binomial())
# Fine, and I can inspect the model. No predictor has std. error > 1.05

# Now if I move the last variable (or any of the last three, for what I've tested) to
# be the first in predictor... 
lModel.3 <- glm(mob_change ~ wh_change + gender + education + start_age + income + dist_change + lu_change + dou_change + marriage + student2work,
            data = regression_data, 
            family = binomial())

Error in separator(X, Y, purpose = "find") : 
  unexpected result from lpSolveAPI for primal test
nazareno
  • 377
  • 1
  • 10
  • 1
    `safeBinaryRegression` changes the `glm` function from `stat` to test for separation first & it's this test that's giving an error message for some reason - not even telling you whether there's separation or not. Of course the order of the predictors is irrelevant to the question. – Scortchi - Reinstate Monica Feb 12 '15 at 16:37
  • What are the form of your independent variables? All categorical, all continuous, or a mix? – Andrew M Feb 12 '15 at 17:11
  • All but `start_age` are categorical, Andrew – nazareno Feb 19 '15 at 14:27

1 Answers1

2

The order the predictors are entered into the model is of course irrelevant to the question of whether there's separation in the data. The safeBinaryRegression package† masks the usual glm function from the stats package, which fits generalized linear models, so that, for logistic regression, glm uses a linear programming algorithm to check for both complete & quasi-complete separation before trying to fit anything. If it finds separation it reports

Separation exists among the sample points.

or

The following terms are causing separation among the sample points:

depending on whether you've asked it just to test for separation or to find the predictors causing it. Otherwise it reports nothing.

Unexpected result from lpSolveAPI for primal test

,however, is a software error message, not a statistical one. You could perhaps try on a machine with more memory, but it's probably safe to trust the results from when you didn't get an error. Using stats:::glm (i.e using the glm function from stats when safeBinaryRegression's loaded) should reach the same results regardless of the order of predictors; it will often report non-convergence or predicted probabilities of nought or one in cases of separation.

Multicollinearity among the predictors is another issue entirely. Generalized variance inflation factors (see the vif function&ddagger; from the car package) are useful for assessing its extent in models with more than one degree of freedom per predictor.

† Konis (2007), "Linear programming algorithms for detecting separated data in binary logistic regression models", DPhil., U. Oxf.

&ddagger; Fox & Monette (1992), "Generalized collinearity diagnostics", JASA, 87, pp178–183.

Scortchi - Reinstate Monica
  • 27,560
  • 8
  • 81
  • 248
  • 1
    I am confused about whether separation is worth special treatment. Regression coefficients that go to $\infty$ are correct, and provide the correct predictions. It is just that their standard errors are useless and one must use the likelihood ratio $\chi^2$ test instead of Wald tests in this situation. – Frank Harrell Feb 13 '15 at 13:01
  • @FrankHarrell: I listed some of the special treatments [here](http://stats.stackexchange.com/questions/11109/); perhaps "Do nothing" should've been at the top of the list. I suppose the "fixes" all involve priors, explicit or implicit, that cast doubt on very high coefficient estimates, & when that's reasonable, better estimates & predictive performance are to be expected. – Scortchi - Reinstate Monica Feb 13 '15 at 14:30
  • 2
    I think the use of prior information is a great idea. It's just that ignoring prior information (as we often do when there is not complete separation) doesn't hurt as much as some think. – Frank Harrell Feb 13 '15 at 19:00
  • I've tried the `safeBinaryRegression`, and it does not complain. I was suspecting this had something to do with separation by inspecting the part of the source code where the message comes from. But I indeed agree this seems like an error in the optimization routine, and not in the statistics. It is just unfortunate I cannot find other people with the same error anywhere... – nazareno Feb 19 '15 at 14:32