17

Can a Random Forest be trained to appropriately predict count data? How would this proceed? I have quite a extensive range of values so classification doesn't really make sense. If I would use regression would I simply truncate the results?

I'm quite lost here. Any ideas?

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
JEquihua
  • 3,525
  • 2
  • 24
  • 44
  • 1
    can't you use poisson regression? – R J Feb 25 '13 at 05:54
  • I wanted to use something non-parametric. Don't really remember poisson regressions assumptions but I'm pretty sure one of them is that the observations are independant and that is deeply not fulfilled here. Could this affect me much? – JEquihua Feb 25 '13 at 13:44
  • 2
    Have you simply tried making RF regression (possibly also in log)? It may just work good enough. –  Feb 25 '13 at 17:39
  • 1
    Have not. But that was my first instict. Log or square root transformation. But I wanted to see if anyone had any experience on this. – JEquihua Feb 25 '13 at 20:23
  • 1
    I tried just doing regression on the response, the log(response) and sqrt(response) and got nothing good. I think the problem is more that my independant variables done explain the response. Oh well. – JEquihua Mar 03 '13 at 00:44

3 Answers3

10

There is a R package called mobForest which can fit a real random forest for count data. It is based on mod() (model-based recursive partitioning) in the party package. It performs Poisson regression if the family argument is specified as poisson(). The package is no longer in the CRAN repository, but formerly available versions can be obtained from the archive.

If you are not restricted to random forest / bagging, a boosting version is also available for count data. That is, gbm (generalized boosted regression models). It can also fit a Poisson model.

Randel
  • 6,199
  • 4
  • 39
  • 65
7

I see a few possibilities.

  • You could bin the response into a few arbitrary categories and use a classification tree
  • If the counts are typically very low, 0, 0, 0, 1, 0, 3, 0, 2, you could treat each integer count as a class and again use a classification tree (probably not your case). In these cases, it's going to be harder to get a high variance explained type metric as opposed to continuous regression.
  • If the counts are not typically low and there is a lot of variation, I'd just go for it with a regression tree. Using poisson regression over linear regression, for instance, is only gravy when it comes to getting a good linear predictor. If you're not seeing good predictive power with the random forest, then I doubt a fancier model that specifically accommodates count data is going to do a lot for you.

Update (2020-12-11) Since writing this answer, a Kaggle contest (the M5 competition) showed me a situation where using Poisson loss in a LightGBM framework did really well for retail sales data with low counts. I don't know how much better it did than mean squared error but many of the public notebooks were using it, and it was an easy switch. I don't think my second bullet is a very good idea but I'll leave it up.

Ben Ogorek
  • 4,629
  • 1
  • 21
  • 41
4

Well, its not random forest, but CatBoost supports a poisson loss function that could be used for count regression with boosted trees:

https://tech.yandex.com/catboost/doc/dg/concepts/loss-functions-docpage/

Udi
  • 69
  • 4
  • The tools keep coming! I updated my answer to say a similar thing about LightGBM, which people were using in the recent M5 competition with Poisson loss. – Ben Ogorek Dec 11 '20 at 14:27