1

In several posts, such as Is centering a valid solution for multicollinearity?, it states that centering doesn't solve multicollinearity because "it's a linear transformation."

I just made up a simple example. Consider $$ X = \begin{bmatrix} 1 & 2\\ 0 & 0 \end{bmatrix} $$ Clearly, the columns are linear dependent here, hence we have multicollinearity. If you were to center $X$, you get $$ X = \begin{bmatrix} 0.5 & 1\\ 0 & -1 \end{bmatrix} $$ Now the columns are no longer linear dependent, and the data matrix is full column rank. So doesn't centering, at times, solve multicollinearity?

roulette01
  • 271
  • 1
  • 7
  • 5
    You misinterpret the meaning of "it's a linear transformation:" in the context, the point was it's a linear transformation *of a single variable at a time.* Any degree of multicollinearity, short of perfect linear dependence, *always* can be removed with a suitable linear transformation *of all the variables simultaneously.* Your example is incorrect: the centered model matrix is $\pmatrix{1/2&1\\-1/2&-1}$ which is still singular. – whuber Jul 10 '20 at 21:14
  • @whuber Are you saying that if there's perfect multicollinearity, then suitable linear transformations of all variables can no longer remove the multicollinearity? Also, if it was the linear transformation of a single variable at a time, wouldn't the matrix actually be (assuming we center the second variable only) $$ \begin{bmatrix} 1 & 1 \\ 0 & =1 \end{bmatrix} $$? In which case would be full rank. – roulette01 Jul 10 '20 at 21:39
  • 3
    Centering is allowable only when the range of $X$ includes the constant vector. (That's because centering is a linear combination of a column vector and the constant vector and that will leave the model unaffected only when the constant vector is already in the model.) That's not the case with your $X,$ so it's an invalid example. As far as the linear transformation claim goes, that's basic linear algebra: a linear transformation can never increase the dimension of a subspace. Perfect collinearity means the image of $X$ is a subspace of smaller dimension than the number of variables, *QED.* – whuber Jul 10 '20 at 21:48

0 Answers0