1

My predictors are all categorical values but the dependent is numerical. How can I eliminate dummies if I use a linear regression model? The values are tough to solve with backward elimination; should I try some other regression?

gung - Reinstate Monica
  • 132,789
  • 81
  • 357
  • 650
ankittheguru
  • 11
  • 1
  • Is the problem that $X$ doesn't have full rank if you code every categorical as dummy variables? You can let your regression package handle the situation (dropping variables or using the pseudo inverse) do it manually, or try coding ordered categorical with a monotone function instead of with dummies, if that makes sense. – CloseToC Sep 01 '19 at 22:41
  • 1
    You can use Decision Tree or Random Forest. – Vitali Avagyan Sep 01 '19 at 21:52
  • thanks, any way to solve it through linear regression ? – ankittheguru Sep 01 '19 at 22:14
  • Do I understand correctly that you want to only use linear regression with no dummy variables? – Vitali Avagyan Sep 01 '19 at 22:27
  • 3
    What do you mean by "eliminate dummies"? Do you mean you want to represent your variables through some means other than dummy codes, or do you mean how can you effectively drop some variables when the variables are represented by dummy codes? Are these categorical variables multi-level, or all binary? Are they ordinal in nature or purely nominal? (You shouldn't use backwards elimination either way...) – gung - Reinstate Monica Sep 01 '19 at 23:46
  • @gung ya some are multi level and some binary, they are nominal in nature. If not backward elimination then what would you suggest ? – ankittheguru Sep 02 '19 at 02:06
  • 2
    There typically isn't actually any need to select variables. People only think there is. (It seems like a very intuitive assumption.) You might read through our threads categorized under [tag:feature-selection] &/or [tag:model-selection]. Generally, sort the results by votes & start reading. To understand some of the problems with stepwise selection routines see [1](https://stats.stackexchange.com/q/20836/7290), [2](https://stats.stackexchange.com/q/179941/), [3](https://stats.stackexchange.com/q/115843/), & [4](https://stats.stackexchange.com/q/6688/7290). – gung - Reinstate Monica Sep 02 '19 at 02:57

0 Answers0