1

Folks, In linear regression, I am looking to understand why the coefficients of a given independent variable (HS_ENGL in this example) would change as other independent variables are added (SAT_VERB SAT_QUAN).

So if I create a model with just HS_ENGL as the predictor, it comes up as significant. As I add additional variables, its coefficient drops + it becomes less significant.

I think if the 3 independent variables had been perfectly orthogonal, then HS_ENGL would have remained the same through the models. So the first thing I did was compute the VIF and got the following. All low values.

HS_ENGL SAT_VERB SAT_QUAN 1.839065 1.300852 1.918336

Next I plotted the correlation between them as below; focus on bottom 3 variables. Hence despite the fact that the VIF were low, the variables are correlated as shown below.

Is this the reason for change in the coefficient of HS_ENGL? Also could there be other possible reasons like 'interaction terms'?...I have not learned about these in any detail yet.

Thanks!

enter image description here

Anand
  • 11
  • 1
  • 3
    The reason is colinearity. Adding interaction term also can change the estimate the parameters, but it is go through the change of colinearity structure. If interaction is orthogonal to other covariates, interaction will have no such effect. – user158565 Oct 21 '18 at 17:24
  • This question is very similar to this question. https://stats.stackexchange.com/questions/439781/gam-interactions-individual-and-combined-interactions-are-different – eric_kernfeld Dec 08 '19 at 22:13
  • Also very similar to this question. https://stats.stackexchange.com/questions/61506/nonsignificant-interaction-still-causes-main-effect-to-flip – eric_kernfeld Dec 08 '19 at 22:15

0 Answers0