24

Does the R caret package cross-validate over both alpha and lambda for the glmnet model? Running this code,

eGrid <- expand.grid(.alpha = (1:10) * 0.1, 
                     .lambda = (1:10) * 0.1)

Control <- trainControl(method = "repeatedcv",repeats = 3,verboseIter =TRUE)

netFit <- train(x =train_features, y = y_train,
          method = "glmnet",
          tuneGrid = eGrid,
          trControl = Control)

The training log looks like this.

Fold10.Rep3: alpha=1.0, lambda=NA

What does lambda=NA mean?

Comp_Warrior
  • 2,075
  • 1
  • 20
  • 35
mrquestion
  • 273
  • 1
  • 2
  • 7

2 Answers2

21

Old question, but I recently had to deal with this problem and found this question as a reference.

Here is an alternative approach:

The glmnet vignette (https://web.stanford.edu/~hastie/glmnet/glmnet_alpha.html) specifically addresses this issue, recommending to specify the cross validation folds using the foldids argument and validate $\lambda$ over a grid of $\alpha$. This uses the same cv folds to validate $\lambda$ for each $\alpha$ in your grid.

The reason this may be superior to validating $\alpha$ and $\lambda$ simultaneously is that cv.glmnet validates $\lambda$ using a 'warm start' to select $\lambda$ rather than just randomly selecting $\lambda > 0$ i.e. speeding up validation and improving likelihood of have the optimal $\lambda$ in your grid (since fine grids are more computationally expensive).

Redeyes10
  • 461
  • 3
  • 5
18

train does tune over both.

Basically, you only need alpha when training and can get predictions across different values of lambda using predict.glmnet. Maybe a value of lambda = "all" or something else would be more informative.

Max

topepo
  • 5,820
  • 1
  • 19
  • 24