4

LASSO regression penalizes coefficients in regression to at most zero. Likelihood ratio tests tells us whether the nested or full model is better.

I used likelihood ratio tests during regression analysis to figure out how to make the most parsimonious models.

I heard Lasso can shrink variables coefficients to zero similar to how likelihood ratio tests tell me I can reduce full models to simpler ones but what are the advantages of using likelihood ratio tests to build more parsimonious models over the lasso?

I find hitting the command anova in R multiple times to perform likelihood ratio tests very tedious and would rather use lasso to shrink my coefficients automatically.

Is the only advantage of using likelihood ratio tests over LASSO that I wouldn't need to cross-validate with the former?

1 Answers1

4

First of all, the LRT is not really a good measure for variable selection. It is intended to compare nested models for evaluation of a hypothesis. If you use the LRT to do variable selection for 10 variables, then you are almost certainly going to make a false positive.

If you want to do a form of variable selection, LASSO seems to be the way to go and is recommended by a lot of applied practitioners.

Demetri Pananos
  • 24,380
  • 1
  • 36
  • 94
  • Why doesn't most software packages include the LASSO? In the NLME package in R there's no lasso, the GEE package doesn't have a LASSO, the lme4 package has no lasso? Everyone I know of using those package likelihood ratio tests to do variable selection. –  Jan 16 '20 at 17:55
  • 1
    Because LASSO biases the coefficients. Those other libraries are for inference, in which you don't want biased estimates. – Demetri Pananos Jan 16 '20 at 17:56
  • Is there a best solution for unbiased variable selection for inference purposes? –  Jan 16 '20 at 17:59
  • 5
    Theoretical knowledge of the problem. You can't automate scientific thinking. – Demetri Pananos Jan 16 '20 at 18:01