1

I have two features for a binary classification problem which are highly positively correlated. (0.79) But when I build a logistic regression classification then I see their weights are opposite in sign. This gives different insights on the response variable based on these features.

Why is that?

Please note there is no correlation in any other features. If I drop one of these features and keep only one of them in the model, still the sign is different for the feature kept separately.

Is this multicollinearity? I mean it is to some extent but then dropping one of the feature should then give sign of individual retained feature same right?.

How to rectify this?

Baktaawar
  • 11
  • 1

1 Answers1

0

Ordinary Least Squares and logistic regression are related by a positive-monotonic link function so the estimators on a linear-probability model and a logistic regression will have the same signs. So you can look at a discussion when asked of OLS (of which there is a lot of literature).

There is a really good discussion on the topic at this question: Regression coefficients that flip sign after including other predictors

Community
  • 1
anguyen
  • 101
  • 2