Can regularization path reveal collinearity?

Question

I know that the penalized Regression is robust to collinearity. I also found a page at scikit-learn : Plot Ridge coefficients as a function of the regularization says it "shows the effect of collinearity in the coefficients of an estimator."

Here is what beats me. There are pattern in heavy wave, or in mild slope; some variables are in same trend, others not.
How do I observe the effect of collinearity from the regularization path plot?
Also does L1 and L2 regularizer has different pattern for collinear variables?
Or detect collinearity using path plot is bad idea?

I believe my answer here at least partly addresses this question: http://stats.stackexchange.com/questions/154825/what-to-conclude-from-this-lasso-plot-glmnet/154853#154853 — Matthew Drury, Mar 30 '16 at 16:20

Analyst · Answer 1 · 2015-03-03T13:38:57.040

1

I would say that change of sign for regression coefficient as alpha goes from K to zero is such indicator.

EDIT:

Huge standard errors and change of signs when omitting/adding variables is a symptom of problems caused by multicollinearity. It is almost impossible to estimate marginal effect from the variables having these problems when they are so strongly related.

I would say this path plot can show which are those variables to which you have to put attention.

edited Mar 03 '15 at 13:38

answered Mar 03 '15 at 10:26

Analyst

2,527
10
11

1

Can you expand on your answer? – Glen_b Mar 03 '15 at 11:10
Added more explation on the issue – Analyst Mar 03 '15 at 13:39
1

yes, but the L2 path do nothing with add/omitting variables. Or are you talking about LASSO path? the large change at kink is hint of collinearity? – WeiChing 林煒清 Mar 04 '15 at 04:14

Can regularization path reveal collinearity?

1 Answers1