I am attempting to run xgboost on response data ( Whether a product is sold or not).However, I have a feature 'number of employees' which is highly skewed.
I'm thinking of convert this continuous into categorical by binning. But I'm not sure about this step.
My question is what is the best approach when dealing with such situation (Skewed data,xgboost)
Thanks in advance !
I would leave the data as is, unless you have some domain knowledge which makes a categorical interpretation sensible. There is no data-independent best approach for this though, you can't know for sure which feature transformations will help without experimenting.
Xgboost (at least the default tree booster) when using the "exact" split finding method will consider every possible split point when building a tree. So you should not be too worried about how the model will handle skewed data, the predictions will be almost (*) invariant to monotonic transformation anyway.
The other thing you should know is that for computational purposes xgboost already has some heuristics for split finding which amount to binning the data before searching for a split (though it won't treat the bins as categorical).
https://xgboost.readthedocs.io/en/latest/treemethod.html
*: I say almost invariant because xgboost will use the midpoint between two training samples as its split point, which will be affected by monotonic transformations.
xgboost works perfectly fine with skewed input data, even large amount of outliers. One of the reason is because it is a tree based model and tree is relatively robust to skewed data and outlier.
In fact, many xgboost users even do not check the distribution of the input. And the xgboost can have thousands of columns as inputs.
In most cases, if you are focusing on accuracy, do not bin the continuous data. Some related posts can be found here.
When should we discretize/bin continuous independent variables/features and when should not?
