1

I am trying to adjust the penalty $\lambda$ in group lasso regression, but I have no idea about it. Just to clarify, group lasso regression is a kind of multiple linear regression which use penalties on estimated coefficients to keep them small. Also, it tries to assign same coefficients to variables which are in the same group.

Is there any theory or rule about maximum and minimum value of $\lambda$ based on input and response? I think the rule of $\lambda$ in lasso works for group lasso as well, so it is helpful.

I need an automatic procedure to determine the minimum and maximum value of penalty because I have more than 10 thousand response variables which regress on more than 500 independent variables.

Sycorax
  • 76,417
  • 20
  • 189
  • 313
user4704857
  • 502
  • 3
  • 12
  • The maximum is discussed here: https://stats.stackexchange.com/questions/210040/, and the minimum is discussed here (despite there not being an answer): https://stats.stackexchange.com/questions/289075/ – user795305 Jul 14 '17 at 18:05

1 Answers1

-1

There is no minimum and maximum value as such. If the penalty becomes too large the LASSO choses none of the variables and hence is too extreme. If the penalty is too small it choses all of the variables and hence there is no regularization. I use cv.glmnet in the glmnet package. It uses CV and tells you what the best $\lambda$ will be (in fact, it insists that unless absolutely necessary, you do not provide your own sequence of $\lambda$)

Sid
  • 2,489
  • 10
  • 15