lambda.min in lasso for correlated variable selection

Question

Lasso uses cross-validation to determine both the number of included predictors and the degree of shrinkage to avoid over-fitting. I have used the glmnet package to do this.

    fit=glmnet(x,y)
    cvob1= cv.glmnet(x,y) 
    lambda1=cvob1$lambda.min  
    cvob2=cv.glmnet(x,y)
    lambda2=cvob2$lambda.min

There is a high possibility that lambda1 and lambda2 are different. I need to get a proper lambda.min for the next step to obtain fitted values for the sparse solution to the model:

coef(fit,s=cvob1$lambda.min)

How can I decide which lambda to use? Should I repeat the cv.glmnet() for many times? How many time should I go for?

You have to run cv.glmnet many times, follow the below answer: https://stats.stackexchange.com/questions/97777/variablity-in-cv-glmnet-results/173895#173895 — forever, Mar 15 '18 at 07:20

score 0 · Accepted Answer · answered Mar 14 '18 at 19:44

0

CV is a random process, which is why you are getting different lambda.mins. But they should be close to eachother. If you want a singular, reproducible lambda, try setting a seed:

set.seed(1234)

answered Mar 14 '18 at 19:44

Tim Atreides

708
3
6

would different lambdas affect my next step: coef(fit,s=cvob1$lambda.min) – ay__ya Mar 14 '18 at 19:52
Yes, since different lambdas would yield different coefficients. (Remember large $\lambda$, small $\|\beta\|$). – Tim Atreides Mar 15 '18 at 11:13
setting a seed for cv.glmnet won't produce identical results unless the foldid's are set manually – David Feb 24 '21 at 21:21

lambda.min in lasso for correlated variable selection

1 Answers1