15

It's been a while since I looked at partitioning trees. Last time I did this sort of thing, I like party in R (created by Hothorn). The idea of conditional inference via sampling makes sense to me. But rpart also had appeal.

In the current application (I can't give details, but it involves trying to determine who will go to jail among a large sample of arrestees) I cannot use advanced methods like random forests, bagging, boosting etc. - I need an easily explicable rule.

I would also like to have some manual control over which nodes split, as recommended in Zhang & Singer (2010) Recursive Partitioning and Applications. The freeware that comes with that book allows this, but is otherwise rather primitive in its user input.

Any recommendations or suggestions?

Ferdi
  • 4,882
  • 7
  • 42
  • 62
Peter Flom
  • 94,055
  • 35
  • 143
  • 276

2 Answers2

9

I agree with @Iterator that the methodology is easier to explain for rpart. However, if you are looking for easily explainable rules, party (without bagged trees) doesn't lose anything in regard to explaining the prediction - you still have a single tree. If you are also interested in looking at drivers of the outcome variable (not just pure predictive power) I would still think that party is the way to go - explaining that a decision tree (like rpart) can be quite biased in how it selects which variables are important and how it creates splits. Party uses permutation tests and statistically determine which variables are most important and how the splits are made. So, instead of biased leaning towards categorical variables with many levels, like rpart for example, party uses statistical tests to find the best structure.

B_Miner
  • 7,560
  • 20
  • 81
  • 144
  • 1
    Nice answer. I think that you've hit a very good reason why party is better for an advanced audience, *and* why it's a good idea to educate the audience in order to help them accept the use of party. – Iterator Feb 08 '12 at 18:43
4

[NB: See update 1 below.] I find that the methodology for rpart is far easier to explain than party. The latter, however, is much more sophisticated and likely to give better models. The way I sometimes explain party is to speak of it as basis for producing local linear (or GLM) models. I build up to this by pointing out that the results for rpart are constant across all elements that fall into the leaf node, i.e. the box/region bounded by the splits. Even if there might be improvements via local models, you don't get anything but a constant prediction.

In contrast, party develops the splits to potentially optimize the models for the regions. It is actually using a different criteria than model optimality, but you need to gauge your own capacity for explaining the difference to determine whether you can explain it well. The papers for it are pretty accessible for a researcher, but may be quite challenging for someone not willing to consider simpler methods like random forests, boosting, etc. Mathematically, I think that party is more sophisticated... Nonetheless, CART models are easier to explain, both in terms of methodology and results, and these provide a decent stepping stone for introducing more sophisticated tree-based models.

In short, I would say that you have to do rpart for clarity, and you can use party for accuracy / peformance, but I wouldn't introduce party without introducing rpart.

Update 1. I based my answer on my understanding of party as it was a year or two ago. It has grown up quite a bit, but I would modify my answer to say that I'd still recommend rpart for its brevity and legacy, should "non-fancy" be an important criterion for your client/collaborator. Yet, I would try to migrate to using more functionality from party, after having introduced someone to rpart. It's better to start small, with loss functions, splitting criteria, etc., in a simple context, before introducing a package and methodology that involve far more involved concepts.

Iterator
  • 2,294
  • 1
  • 15
  • 22
  • 2
    I think you're confusing up a little bit about what the `party` package can do. The pure `party` function makes just a single simple tree like `rpart` with majority voting in the leaves. The `mob` function in `party` is what builds trees with more complex models in the leaves (and chooses the splits based on parameter instability.) – Shea Parkes Feb 08 '12 at 18:24
  • 1
    @SheaParkes You're right. It's been awhile, and I'm not sure if I only used `mob` or if the rest of the package has grown quite a bit - I don't remember seeing random forests before, for instance. I'll revise my answer... – Iterator Feb 08 '12 at 18:37
  • 3
    And actually, I'd forgotten a bit too. It's `ctree` to make a single tree, `cforest` to make a random forest and `mob` to make the model based leaves. And fyi, cforest is fun, but horribly slow to predict with. – Shea Parkes Feb 08 '12 at 21:08
  • I will look into **mob**, I don't think it existed last time I used **party**. The forest applications aren't for me, this time. – Peter Flom Feb 08 '12 at 23:10
  • @PeterFlom I think `mob` may have been there since the beginning, or at least came after `ctree`, I suppose. It's been around since 2009 or earlier. Anyway, just goes to show that we can all learn something new on SE. :) – Iterator Feb 09 '12 at 00:03