Uniform implementation of Random Forest and Adaboost

Question

If I have

a set RF of Decision Trees trained using a Random Forest algorithm and

a set AB of Decision Trees (Stumps), trained using Adaboost

do I see it correctly that I can uniformly implement both Ensemble methods, by something along the line of

  double ensemble(Trees, weights) {
      dec = 0
      for T in Trees:
        dec += weights[T] * T.getDecision() //-1 for False, +1 for True
      return dec
  }

and call it for the Random Forest using

ensemble(RF, [1,1,...])

and for Adaboost using

ensemble(RF, w)

where w is the weight vector obtained form the Adaboost training.

I.e., after training the only difference between Adaboost and RandomForest is that Adaboost uses a weighted sum of the decision but RandomForest doesn't?

score 3 · Accepted Answer · edited Apr 13 '17 at 12:44

I believe that should be possibly. Basicly you need both adaboost and random forest to not only classify, but also quantify some kind of certainty. For RF, probabilistic voting is the average vote over trees. I see adaBag package for R outputs a similar probabilistic measure.

citing from package manual, "predict.boosting: Predict from a fitted boosting object."

prob a matrix describing, for each observation, the posterior probability or degree of support of each class. These probabilities are calculated using the proportion of votes in the final ensemble.

Use ROC plots to check if combined ensembles actually are better.

Uniform implementation of Random Forest and Adaboost

1 Answers1