According to Random Forest algorithm, tress are not supposed to be pruned intuitively, don't bagging on pruned models give a better final model than the one without pruning?
Asked
Active
Viewed 37 times
0
-
The question you tagged says, Bagging and de-correlated will take care of Pruning and I understand that. The one I am asking is a bit different. – Jason Mathews May 10 '19 at 18:21
-
Since I'm unable to write an answer (due to the question being locked), I will leave a short explanation as a comment. Individual trees are pruned in order to reduce their variance and thus overfitting, however this in turn increases the model bias (see bias-variance trade-off). When you add multiple trees, the variance of the individual tree outputs is not added, while the bias is added up. Therefore, assuming totally uncorrelated trees, averaging the tree outputs can reduce the variance by a factor of n, where n is the number of trees. – Scholar May 13 '19 at 13:11
-
Since the bias is added up, averaging over multiple trees has no effect and the bias remains the same. Therefore, random forests aim to minimize the bias of the individual trees at the expense of high variance and thus the trees are build to node purity. – Scholar May 13 '19 at 13:14