2

I am trying to fit a multinomial logit model using glmnet. I have a few questions:

  1. How is the baseline category specified?

  2. Looking at the model coefficients using coef.glmnet, I'm thinking that many are given by a dot. I assume this means the coefficient was set to zero by LASSO, so the variable was dropped. However, I'm finding that variables are dropped this way even when lambda=0, so there is no regularizing term. Can someone explain what's going on here?

  3. Sometimes I find that a variable is dropped in some but not all of the logits. How should this be interpreted?

Here is a working example: 

library(glmnet)
data(iris)
X <- model.matrix(Species ~., data=iris)
y <- iris$Species
fit <- glmnet (X, y, family="multinomial")
coef(fit, s=0)
kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
abayesed
  • 143
  • 1
  • 7

1 Answers1

1

Q1: glmnet do not use (or need) a baseline category. The penalization takes care of the linear dependence, no matrix is inverted during the fitting! Furthermore, it would be an error to use a baseline category, because different choices would lead to fitted models giving different predictions. See discussion in Dropping one of the columns when using one-hot encoding

Q2: Not sure about this, probably has to do with details about the algorithm (which is iterative, not exact). Leaving that one to others.

Q3: The usual lasso do not know about different dummys belonging to the same categorical variable, so it might choose to set different variables to zero. To avoid that, use the group lasso. See Why use group lasso instead of lasso?

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467