19

I'm building a logistic regression in R using LASSO method with the functions cv.glmnet for selecting the lambda and glmnet for the final model.

I already know all the disadvantages regarding the automatic model selection but I need to do it anyway.

My problem is that I need to include factor (categorical) variables in the model, is there any way to do it without creating a lot of dummy variables? This variables are almost all strings and not numbers.

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
Dan
  • 375
  • 2
  • 3
  • 7

1 Answers1

32

glmnet cannot take factor directly, you need to transform factor variables to dummies. It is only one simple step using model.matrix, for instance:

x_train <- model.matrix( ~ .-1, train[,features])
lm = cv.glmnet(x=x_train,y = as.factor(train$y), intercept=FALSE ,family =   "binomial", alpha=1, nfolds=7)
best_lambda <- lm$lambda[which.min(lm$cvm)]

alpha=1 will build a LASSO.

Romain
  • 1,471
  • 9
  • 9
  • 3
    +1 Great answer! May I ask why you or anyone why one uses intercept=FALSE? – Erosennin Jan 24 '19 at 08:49
  • This seems to fail when there are two categorical variables: I'm rightly getting L1 columns if var1 has L1 levels, but L2-1 columns for var2 (which has L2 levels). – Peter Straka Oct 15 '19 at 14:33
  • 1
    @Peter Straka: sum(over L1 dummies for var1) = 1 for all records, and sum(over L2 dummies for V2) = 1 for all records, so the L1 dummies for var1 and the L2 dummies for var2 are linearly dependent. At least one of the dummies L2 dummies for var2 is redundant (for the purpose of building a linear model). – VictorZurkowski Oct 23 '19 at 18:20