WEKA: Visualize combined trees of random forest classifier

Question

I have a small data set consisting of 385 entries and around 200 attributes. Because I want to apply attribute selection and because of the limited size of my data set, I got the advice to use the random forest classifier, because it got attribute selection build in and does not require an extra training set to determine the attributes to be used.

My question is if it is also possible in WEKA to visualize the final tree of the random forest classifier, so that I can see which attributes are eventually selected? If I set the debug option, I only see the intermediate trees.

Can I for example also determine the attributes that are eventually selected using the random forest classifier in the attribute selection tab of the WEKA explorer, or will this result in other attributes being selected?

Answer 1

Random forests does not generate a final tree. There are however ways to get insights from a random forest:

• Obtaining knowledge from a random forest
• Ideas for outputting a prediction equation for Random Forests ( in particular have a look at point 3: Combining Multiple Models, where for example you might build a tree on top of random forest outputs) I implemented a version of this method in WEKA, however it might be old: file

I just found this publication: Eibe Frank, Michael Mayo, and Stefan Kramer. Alternating model trees. In Proc 30th ACM Symposium on Applied Computing, Data Mining Track, pages 871-878. ACM Press, 2015.

try to have a look.

Moreover, many attributes will be selected in random forest. One way to see which attributes are selected more often is to use random forest variable importance. Unfortunately I don't think it is implemented in the standard WEKA RandomForest.

Answer 2

It is possible to visualize tree for random forest as well. You can use weka 3.7.7. When you are selecting RandomForest select printTrees option true to visualize the tree

Hope this helps