I am looking for the best $\alpha$ (=ratio between L1 and L2 penalty) and $\lambda$ (=penalty strength) for my elastic net regression model, using the R package glmnet
. I have found my $\alpha$ by cross-validation using cv.glmnet
& fixed foldids, following the Glmnet vignette and Stackexchange question (see code below).
Short version: I don't know if I have to tune for $\lambda$ seperately using the tuned $\alpha$ OR if I should directly extract the lambda.min
from the same cv.glmnet
object?
Longer version: After I tune for my $\alpha$ following this code:
#library("glmnet")
#Tune for alpha (L1/L2):
foldid=sample(1:10,size=length(y),replace=T)
alphalist <- seq(0,1,by=0.1)
cv_alpha <- lapply(alphalist, function(a){
cv.glmnet(x, y, foldid=foldid, alpha=a, family="gaussian")
})
for (i in 1:11) {print(min(cv_alpha[[i]]$cvm))}
#cv_alpha[[9]] results in the minimum cvm and is chosen for further analyses
...should I directly extract the lambda.min
from the same cv.glmnet
object I used for tuning for $\alpha$ by calling it with cv_alpha[[9]][["lambda.min"]]
when extracting the coefficients, as in the code below:
##first run the glmnet
set.seed(1010)
elast = glmnet(x, y, alpha=0.8, family="gaussian")
## extract coefficients at specific lambda as determined in cv_alpha
beta_elast <- coef(elast, s = elasticnet[[9]][["lambda.min"]])
OR is it more advisable to cross-validate for the $\lambda$ seperately with a new cv.glmnet
at the determined $\alpha$?
lambdas <- 10^seq(2, -3, by = -.1)
cv_lambda <- cv.glmnet(x, y, alpha = 0.8, lambda = lambdas)
beta_elast <- coef(elast, s = cv_lambda$lambda.min)
Edit: Added a short version of the question further up and edited what I mean with $\alpha$ and $\lambda$