4

In the language R there is the possibility to estimate the best $\lambda$ for an elastic net model via a cross-validation, having set the value of $\alpha$. However, being $\alpha$ defined as $\lambda_2/(\lambda_1+\lambda_2)$, where $\lambda_1$ and $\lambda_2$ are the coefficients of the penalizations for the ridge and lasso terms, how is $\lambda$ defined?

Pippo
  • 587
  • 1
  • 6
  • 14

1 Answers1

13

You're confused; $\alpha$ and $\lambda$ are totally different.

$\alpha$ sets the degree of mixing between ridge regression and lasso: when $\alpha = 0$, the elastic net does the former, and when $\alpha = 1$, it does the latter. Values of $\alpha$ between those extremes will give a result that is a blend of the two.

Meanwhile, $\lambda$ is the shrinkage parameter: when $\lambda = 0$, no shrinkage is performed, and as $\lambda$ increases, the coefficients are shrunk ever more strongly. This happens regardless of the value of $\alpha$.

Hong Ooi
  • 7,629
  • 3
  • 29
  • 52
  • 1
    My problem was that in my lecture notes the professor used the notation of $\lambda_1$ and $\lambda_2$ to identify the parameters of the ridge and the lasso regression, and from them he derives the generalization of the elastic net, setting $\alpha$ like in the question. Hence, I thought $\lambda$ of R was related to these parameters! :) – Pippo Jul 24 '13 at 12:24