0

In How to Deal with Multicollinearity?, the top comment in Aaron's answer says "These independent variables are now uncorrelated. Very low eigenvalues also indicate high degrees of multicollinearity in the original data.."

Why are the variables now uncorrelated, and why do very low eigenvalue indicate high degrees of multicollinearity?

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
user5965026
  • 503
  • 2
  • 11
  • 1
    Hmm I'm not sure if that comment on the independent variables being uncorrelated is correct. The principal components corresponding to distinct singular values are orthogonal, which implies linear independency, but if the principal components are not centered, then it doesn't imply that they're uncorrelated. Centered orthogonality implies the variables are correlated, since covariance between $X,Y$ is defined as $E[(X - \mu_X)(Y - \mu_Y)]$. – 24n8 Aug 08 '20 at 18:59
  • My post at https://stats.stackexchange.com/a/74328/919 answers your second question. The first ("why are the variables now uncorrelated?") is basic to PCA; you can find explanations in many PCA-related threads on this site. – whuber Aug 08 '20 at 19:04
  • @whuber I will check it out. As lamanon said above the principal components are orthogonal, but may not be uncorrelated if they're not centered. Are the principal components centered? I know that typically before we perform PCA, we perform centering on the *original* variables, but does this result in the principal components being centered too? If not, then it still may be correlated? – user5965026 Aug 08 '20 at 19:07
  • Yes. The original means are mapped to the principal component space, so the zero mean is preserved. But if you don't center the data before hand, then there's no guarantee of uncorrelation. There is only guarantee of orthogonality of the eigenvectors corresponding to the distinct eigenvalues of the covariance matrix. – 24n8 Aug 08 '20 at 19:55

0 Answers0