2

I am reading the book titled "An Introduction to Statistical Learning" by James et al. There it is mentioned on page 309 that we pick the cost complexity parameter α to minimize the average Mean Squared Prediction Error.

Then on page 326, the following is mentioned:

"We use the argument FUN=prune.misclass in order to indicate that we want the classification error rate to guide the cross-validation and pruning process, rather than the default for the cv.tree() function, which is deviance."

My question is when using cross-validation to choose α, which criterion should we use then? Average Mean Squared Prediction Error or Classification error rate? It seems that at one place it is mentioned that we need to look at the average mean squared prediction error and at another place it is mentioned that we need to use classification error.

Satwik Bhattamishra
  • 1,446
  • 8
  • 24
Anup
  • 135
  • 6

1 Answers1

2

It seems that at one place it is mentioned that we need to look at the average mean squared prediction error and at another place it is mentioned that we need to use classification error.

The reason is that pages 304-310 explain regression trees. That is why it talks about using Mean Squared Prediction Error as the evaluation metric. There is no classification error rate for a regression problem.

Then from page 311-314, it explains about Classification Trees. The excerpt from page 326 which you have quoted is under the section 'Fitting Classification Trees'. That is why it uses Classification error rate. You can use classification error rate to improve your model in case of classification problems and similarly you can use Mean Squared Prediction Error in case of regression problems.

Satwik Bhattamishra
  • 1,446
  • 8
  • 24