Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [elastic-net]

A regularization method for regression models that combines the penalties of lasso and of ridge regression.

A regularization method for regression models that penalizes the size of regression coefficients $\beta_i$ and biases them towards zero. Elastic net includes two penalty terms, one proportional to $\sum |\beta_i|$ and another proportional to $\sum \beta_i^2$. When used alone, these penalty terms lead to Lasso regression and ridge regression respectively.

275 questions
42
votes
4 answers

Ridge, lasso and elastic net

How do ridge, LASSO and elasticnet regularization methods compare? What are their respective advantages and disadvantages? Any good technical paper, or lecture notes would be appreciated as well.
user3269
  • 4,622
  • 8
  • 43
  • 53
42
votes
2 answers

What is elastic net regularization, and how does it solve the drawbacks of Ridge ($L^2$) and Lasso ($L^1$)?

Is elastic net regularization always preferred to Lasso & Ridge since it seems to solve the drawbacks of these methods? What is the intuition and what is the math behind elastic net?
GeorgeOfTheRF
  • 5,063
  • 14
  • 42
  • 51
35
votes
2 answers

Why is lambda "within one standard error from the minimum" is a recommended value for lambda in an elastic net regression?

I understand what role lambda plays in an elastic-net regression. And I can understand why one would select lambda.min, the value of lambda that minimizes cross validated error. My question is Where in the statistics literature is it recommended to…
jhersh
  • 353
  • 1
  • 4
  • 5
33
votes
1 answer

Why does glmnet use "naive" elastic net from the Zou & Hastie original paper?

The original elastic net paper Zou & Hastie (2005) Regularization and variable selection via the elastic net introduced elastic net loss function for linear regression (here I assume all variables are centered and scaled to unit variance):…
amoeba
  • 93,463
  • 28
  • 275
  • 317
31
votes
2 answers

Choosing optimal alpha in elastic net logistic regression

I'm performing an elastic-net logistic regression on a health care dataset using the glmnet package in R by selecting lambda values over a grid of $\alpha$ from 0 to 1. My abbreviated code is below: alphalist <- seq(0,1,by=0.1) elasticnet <-…
RobertF
  • 4,380
  • 6
  • 29
  • 46
31
votes
1 answer

Feature selection & model with glmnet on Methylation data (p>>N)

I would like to use GLM and Elastic Net to select those relevant features + build a linear regression model (i.e., both prediction and understanding, so it would be better to be left with relatively few parameters). The output is continuous. It's…
PGreen
  • 565
  • 1
  • 6
  • 11
31
votes
2 answers

Fitting an ARIMAX model with regularization or penalization (e.g. with the lasso, elastic net, or ridge regression)

I use the auto.arima() function in the forecast package to fit ARMAX models with a variety of covariates. However, I often have a large number of variables to select from and usually end up with a final model that works with a subset of them. I…
Zach
  • 22,308
  • 18
  • 114
  • 158
26
votes
2 answers

Why Lasso or ElasticNet perform better than Ridge when the features are correlated

I have a set of 150 features, and many of them are highly correlated with each other. My goal is to predict the value of a discrete variable, whose range is 1-8. My sample size is 550, and I am using 10-fold cross-validation. AFAIK, among the…
renakre
  • 755
  • 1
  • 9
  • 25
25
votes
1 answer

Bridge penalty vs. Elastic Net regularization

Some penalty functions and approximations are well studied, such as the LASSO ($L_1$) and the Ridge ($L_2$) and how these compare in regression. I've been reading about the Bridge penalty, which is the $\sum \|\beta_{j}\|^{\gamma}$ generalized…
Firebug
  • 15,262
  • 5
  • 60
  • 127
22
votes
1 answer

Is R-squared value appropriate for comparing models?

I'm trying to identify the best model to predict the prices of automobiles, using the prices and features available on automobile classified advertisement sites. For this I used a couple of models from the scikit-learn library and neural network…
Manik
  • 321
  • 1
  • 2
  • 4
22
votes
3 answers

Model stability when dealing with large $p$, small $n$ problem

Intro: I have a dataset with a classical "large p, small n problem". The number available samples n=150 while the number of possible predictors p=400. The outcome is a continuous variable. I want to find the most "important" descriptors, i.e.,…
dimi
  • 321
  • 2
  • 6
22
votes
3 answers

Using regularization when doing statistical inference

I know about the benefits of regularization when building predictive models (bias vs. variance, preventing overfitting). But, I'm wondering if it is a good idea to also do regularization (lasso, ridge, elastic net) when the main purpose of the…
user162381
  • 291
  • 3
  • 8
20
votes
1 answer

Is there a Bayesian interpretation of linear regression with simultaneous L1 and L2 regularization (aka elastic net)?

It's well known that linear regression with an $l^2$ penalty is equivalent to finding the MAP estimate given a Gaussian prior on the coefficients. Similarly, using an $l^1$ penalty is equivalent to using a Laplace distribution as the prior. It's not…
Michael Curry
  • 365
  • 1
  • 6
20
votes
2 answers

Elastic/ridge/lasso analysis, what then?

I'm getting really interested in the elastic net procedure for predictor shrinkage/selection. It seems very powerful. But from the scientific point of view I don't know well what to do once I got the coefficients. What question am I answering? These…
Bakaburg
  • 2,293
  • 3
  • 21
  • 30
18
votes
3 answers

Lasso vs. adaptive Lasso

LASSO and adaptive LASSO are two different things, right? (To me, the penalties look different, but I'm just checking whether I miss something.) When you generally speak about elastic net, is the special case LASSO or adaptive LASSO? Which one does…
Mr Validation
  • 211
  • 1
  • 2
  • 4
1
2 3
…
18 19