1

I'm new to Machine Learning. I'm trying to build a regression model, to predict the price of a cab ride. I have distance, source, destination and other independent variables.

Do I need to encode (one hot encoding) both source and destination variables (categorical) before building a model?

Note: source and destination have the same number of unique elements, and I also thought of dropping source and destination, and keeping only distance. However, some records have different distance values for the same source and destination (because of cab drivers taking different routes).

I'm also worried about the curse of dimensions after encoding. How would you proceed with this type of data?

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
chqdrian
  • 127
  • 4
  • 1
    Sample size? How many sources/ destinations? Maybe have a look at https://stats.stackexchange.com/questions/146907/principled-way-of-collapsing-categorical-variables-with-many-levels, and look at the *fused lasso*. Still there is the choice between coding source and destination separately, or coding the pairs directly, or something else. – kjetil b halvorsen Nov 15 '19 at 14:15
  • sample size - 690,000 unique values in source - 12, destination -12 – chqdrian Nov 15 '19 at 14:27
  • So curse of dimensionality should not be a problem in this case ... – kjetil b halvorsen Nov 15 '19 at 14:42
  • Thanks @kjetilbhalvorsen – chqdrian Nov 15 '19 at 14:59

1 Answers1

1

I suggest you think carefully about the context of the question in practice.

You could model the source and destination as GPS coordinates rather than streets and house numbers. In this case your outcome will be two numbers: the x and the y coordinate. Your approach would require multi-class classification and one way to encode the outcome would be as a one-hot encoded matrix.

If there are many source and destination values, then the curse of dimensionality is a valid concern. Some algorithms are designed to handle many categorical dimensions, e.g. catBoost. The underlying idea here is to replace each level in the categorical variable by a noisy average of the target variable for that level. That turns a categorical variable into a single continuous variable.

The other state-of-the-art approach is use separate input layers for categorical variables in a neural network, a model structure know as embedding (layers). The embedding layer reduces the large dimensionality into a lower dimensional representation that serves as input to the prediction layer/linear model. This has been shown to be effective in a taxi competition on Kaggle

EDIT: Twelve unique source destination are likely not an issue w.r.t dimensionality. The above solutions handle categories in the hundreds or thousands.

Johannes
  • 406
  • 2
  • 10