0

For multiple regression, sometimes highly correlated independent variables do not exhibit multicollinearity measured by Variance Inflation Factor.

What could be the reason/reasons?

Can some one suggest a good reading material also?

user2728024
  • 105
  • 1
  • 1
  • 5
  • How are you measuring the collinearity of the variables in question? – Chris Aug 23 '17 at 02:43
  • Using VIF. updated question. – user2728024 Aug 23 '17 at 05:46
  • 1
    *Inter alia*, please see https://stats.stackexchange.com/questions/34488 and https://stats.stackexchange.com/a/14528/919 (for an explicit example of not-too-high VIFs in the presence of correlation problems in a multiple regression). – whuber Aug 24 '17 at 15:16
  • 1
    Possible duplicate of [Either quadratic or interaction term is significant in isolation, but neither are together](https://stats.stackexchange.com/questions/34488/either-quadratic-or-interaction-term-is-significant-in-isolation-but-neither-ar) – kjetil b halvorsen Jun 17 '19 at 09:47

2 Answers2

1

What do you mean by "high" correlation?

$VIF = \frac{1}{1-R^2_j}$

If there are only 2 independent variables, then this is pretty simple. If the corrleation (r) is 0.8 then $VIF = \frac{1}{1-0.64} = \frac{1}{0.36} = 2.77$ Is 0.6 'high'?

But if there are more variables, then it gets trickier.

Peter Flom
  • 94,055
  • 35
  • 143
  • 276
0

Consider the definition of the VIF: $$ \text{VIF} = \frac{1}{1-R^2_j} $$ Where $R^2_j$ is the coefficient of determination of predictor variable $x_j$. Now $$ R^2_j = 1 - \frac{\sum_i e_i}{\sum_i(y_i-\bar{y})^2} $$ which gets larger as the sum of the residuals (errors) in the numerator gets smaller. A larger $R^2_j$ will in turn cause the VIF to get larger. Geometrically speaking a line that fits snugly between the points of two predictor variables on a graph will have little error and thus a large VIF.

The Pearson correlation is different than fitting a line through data. It measures how well two variables are linearly correlated and does not attempt to find a line that best minimizes the error between it and the data points. The following plot is the best explanation.

VIF Plot

Here we see a strong correlation between the predictor variables $a$ and $b$ and a relatively low VIF. This is because it is difficult to fit a line through this data set but easy to see that the variables are well correlated.

Chris
  • 681
  • 4
  • 13
  • These variables are *not* well correlated. You seem to pull a linguistic sleight-of-hand at the end by changing the sense of "correlated" from "having a high correlation coefficient" to "being clearly related." They are not the same! – whuber Aug 24 '17 at 15:13
  • $-.6$ isn't bad. But let's say the correlation is somewhat low, I do think the example illustrates the idea. I'm not trying to be deceptive. The numbers are there for the reader to interpret regardless of the subjective description. – Chris Aug 24 '17 at 15:19
  • Since OLS regression and VIF are *entirely* based on first and second order moments, you confuse the discussion by introducing data patterns that cannot be discriminated on the basis of these low moments. Your illustration is (at best) tangential to the real issues. – whuber Aug 24 '17 at 15:22
  • I look forward to your elucidating answer :). – Chris Aug 24 '17 at 15:23
  • 1
    I have posted many. For a few, please see the links I mention in a comment to the question. – whuber Aug 24 '17 at 15:25