Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [pruning]

23 questions
30
votes
3 answers

Why Not Prune Your Neural Network?

Han et al. (2015) used a method of iterative pruning to reduce their network to only 10% of its original size with no loss of accuracy by removing weights with very low values, since these changed very little. As someone new to the machine learning…
RoryHector
  • 423
  • 3
  • 5
13
votes
2 answers

How to choose $\alpha$ in cost-complexity pruning?

In the following lectures Tree Methods, they describe a tree algorithm for cost complexity pruning on page 21. It says we apply cost complexity pruning to the large tree in order to obtain a sequence of best subtrees, as a function of $\alpha$. My…
itzjustricky
  • 379
  • 1
  • 3
  • 12
7
votes
2 answers

Neural network's weight reduction

Are there any algorithms/methods for taking a trained model and reducing its number of weights with as little negative effect as possible to its final performance? Say I have a very big (too big) model which contains X weights and I want to cut it…
Mark.F
  • 623
  • 1
  • 5
  • 17
4
votes
1 answer

How to obtain regularization parameter when pruning decision trees?

I'm having trouble understanding exactly how to obtain the regularization parameter when pruning a decision tree with the minimal cost complexity approach. Assume the cost complexity function is represented as $$C(T) = R(T) + \alpha|T|,$$ where…
Jacob H
  • 762
  • 5
  • 17
4
votes
0 answers

Machine Learning and Flow Maximization

Has anyone ever seen machine learning (ML) used to assist a Max Flow algorithm? I have a very large directed graph that has some fractal characteristics, meaning that this large graph can be roughly split into smaller ones. I was wondering if I…
rafbrl
  • 41
  • 1
3
votes
1 answer

Overview of the main methods to prune decision trees

Could someone explain the main pruning techniques for decision trees. So something like the 3 most common techniques with a short explanation of how they work. I have looked online but this, surprisingly, doesnt seem to have been covered anywhere. A…
Trajan
  • 369
  • 4
  • 17
3
votes
1 answer

Pruning in Decision Trees?

Pruning is a technique in machine learning and search algorithms that reduces the size of decision trees by removing sections of the tree that provide little power to classify instances. I know what is decision trees and how it works. I am having…
Pluviophile
  • 2,381
  • 8
  • 18
  • 45
3
votes
1 answer

Avoiding overfitting with linear regression trees

I use regression trees (R package rpart) in my statistical analysis, and have received a critical comment that this method amounts to a "hunting expedition" that will always produce a result ("statistical creep"), but not necessarily one that helps…
robert
  • 686
  • 1
  • 8
  • 18
2
votes
1 answer

Decision Trees: Cost Complexity Parameter and $-\infty$

I am reading the book titled "An Introduction to Statistical Learning with Applications in R" by James et al. On page 326, we perform cross-validation to determine the optimal level of tree complexity (for a classification tree). Here, you can find…
Zachary
  • 135
  • 3
2
votes
0 answers

Determining alpha for pruning trees with cross-validation

following the answer from of Steffen to the question below: How to choose $\alpha$ in cost-complexity pruning? and slide 10 in: https://web.stanford.edu/class/stats202/content/lec19.pdf I'm still unsure about the algorithm to determine the best…
user541057
  • 21
  • 3
2
votes
1 answer

Cost complexity pruning and prediction error

I am reading the book titled "An Introduction to Statistical Learning" by James et al. There it is mentioned on page 309 that we pick the cost complexity parameter α to minimize the average Mean Squared Prediction Error. Then on page 326, the…
Anup
  • 135
  • 6
2
votes
1 answer

Post-Pruning in partykit: the size of mob() tree

I am trying to build a multiple regression model while partitioning my data into subgroups based on additional set of covariates. While I implemented lmtree() or mob() in the "partykit" package, I tried to understand post-pruning strategies using…
sunmee
  • 23
  • 2
2
votes
0 answers

pruning : why if T1 and T2 (2 subtrees) with the same risk imply that one must be a subree of the other

I don't understand the following assertion from "An Introduction to Recursive Partitioning" page 13. If T1 and T2 are sub trees of T with Rα(T1) = Rα(T2), then either T1 is a sub tree of T2 or T2 is a sub tree of T1; hence either |T1| < |T2| or…
André Mayers
  • 21
  • 2
1
vote
1 answer

What is the difference between network sparsification and model pruning

What is the difference between network sparsification and model pruning? I watched USENIX ATC '21 - Octo: INT8 Training with Loss-aware Compensation and Backward Quantization for Tiny (at 01:29sec) where they state them as two different methods to…
Mas A
  • 175
  • 9
1
vote
0 answers

How does cost_complexity_pruning_path in sklearn calculate effective alphas when pruning a decision tree?

I know when pruing DecisionTreeRegressor, we can leverage cost_complexity_pruning_path method to get a list of effective alphas. But how this method calculates the alphas?
Alice Wang
  • 11
  • 1
1
2