1

Let's say I fit logistic regression with 2 predictors (x1, x2) and binary outcome (y). When I fit two separate regression y~x1 and y~x2 coefficients for predictors are significantly different from 0, meaning that their CIs don't contain 1 (for Odds Ratios).

However when I fit logistic regression with both parameters together, y~x1+x2, coefficients are no longer significantly different from 1. Why does it happen? How could this be explained?

Here is the particular example: I have variables x1, x2, and I consider predictors as x1 and x2/x1.

summary(glm(as.formula(data2[,1]~data2[,2]), family=binomial(link="logit"), 
            na.action=na.pass))

gives me:

(Intercept) -1.05526    0.29529  -3.574 0.000352 ***
data2[, 2]   0.24653    0.09686   2.545 0.010917 * 

summary(glm(as.formula(data2[,1]~I(data2[,2]/data2[,3)), family=binomial(link="logit"), 
            na.action=na.pass))

gives me:

(Intercept)               0.43475    0.39187   1.109  0.26725   
I(data2[, 3]/data2[, 2]) -0.06889    0.02554  -2.698  0.00699 **

while regression with both variables, 

summary(glm(as.formula(data2[,1]~data[,2]+I(data2[,2]/data2[,3)), 
            family=binomial(link="logit"), na.action=na.pass))

gives me:

(Intercept)               -0.2278     0.5740  -0.397    0.692
data2[, 2]                 0.1580     0.1065   1.484    0.138
I(data2[, 3]/data2[, 2])  -0.0456     0.0284  -1.606    0.108

Correlation between variables is -0.504 and is significantly different from 0.

Does significantly different from 0 correlation lead to this situation? If so, how could one justify this?

gung - Reinstate Monica
  • 132,789
  • 81
  • 357
  • 650
Oleg
  • 611
  • 4
  • 14
  • 1
    There's lots of ways it could be explained which mostly come down to x1 and x2 are correlated. Flesh out your question with more information about the exact results you have and you'll get a more targeted answer. – John Feb 08 '15 at 19:38
  • What we need to know is if x1 & x2 are uncorrelated. Also what are these? Is there a substantive meaning of x2/x1 that is distinct from the 2 vars? Can x1 be 0, even theoretically (hopefully not)? – gung - Reinstate Monica Feb 08 '15 at 20:21
  • neither x1 nor x2 could be 0. Correlation between x1 and x2 is 0.13, not significantly different from 0. Separate regression with x2, y~x2 didn't give good p-value while x2/x1 did. That's why I've substituted variables and now I have x1 and (x2/x1). Now when I have them, why could this happen?! because of correlation? If so, could you give me a link to the literature where it is discussed why regression with correlated coefficients give non-significant result despite separate regressions are significant – Oleg Feb 08 '15 at 20:28
  • How much data do you have? Did you simply try different combinations of variables & settle on x2/x1, instead of x2, because the former was significant & the latter wasn't? – gung - Reinstate Monica Feb 08 '15 at 20:30
  • My data contains of 64 rows, each of y (0 or 1), x1 and x2 continuous. You're right, I chose combination x2/x1 because it was significant while x2 itself wasn't – Oleg Feb 08 '15 at 21:31
  • Since logistic regression is linear regression at its core, you might want to search for "the multi-collinearity problem" with linear regression. The intuitive reason why the coefficients are no more significant is that if you have perfect correlation between your predictors (i.e., say x1 = x2, and say y = x1), then any coefficient 'a' will satisfy `y = a*x1 + (1-a)*x2`. Since there are many coefficients 'a' that satisfy the relation, the CIs for coefficients tend to be large. – Vimal Feb 08 '15 at 21:49

0 Answers0