4

I want to use LASSO to construct a model and then run a logistic regression on the variables LASSO selects. However, LASSO selects only parts of some categorical variables that I put into it.

Does that mean I should dichotomize the categorical variables and only use those that LASSO selected or should I include the entire categorical variable in my regression model?

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
Paze
  • 1,751
  • 7
  • 21
  • 2
    You should try group lasso. The categorical variable can be selected as a whole using this method – Demetri Pananos Jan 21 '20 at 20:08
  • 2
    Just like running individual testing on coefficients that are really part of a larger categorical variable, it is nonsensical to do so and remove the "parts" that a "test" doesn't "pick" (you first would use a joint test of all coefficients for that variable before considering individual tests, and even still, you wouldn't generally remove portions that aren't "picked" by the test or algorithm). Group LASSO as suggested above would the far superior option over removing/collapsing categories (generally speaking). – LSC Jan 21 '20 at 20:16
  • Would including the entire category with my OP method be superior to group LASSO? – Paze Jan 22 '20 at 06:46

1 Answers1

2

You should use group lasso. When you have a categorical variable, with say, $k$ levels, usually represented in a regression model with $k-1$ dummys, that group of $k-1$ dummys together is the one variable, and should be included or not as a unit. Group lasso does that.

This have been explained many times on this site, for example Can I ignore coefficients for non-significant levels of factors in a linear model? and at your other Q here When finding significance in parts of a categorical variable during stepwise selection, do you include the entire variable?

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467