I need to find out the regions (could be more than 1) where a classifier performs well on training data. I need to somehow store those regions and for two or more classifiers find out the overlaps of regions of different classifiers. How can I implement this in R or any other libraries? The dataset will definitely be multidimensional.
Asked
Active
Viewed 33 times
2
-
1What classifier do you use? It certainly depends on this, for example decision trees have this capability out of the box. – Jakub Bartczuk Sep 05 '17 at 08:18
-
1Also you might want to read on semiparametric regression. – Jakub Bartczuk Sep 05 '17 at 08:18
-
My problem is for general case. The classifier can be of any type. It's a vital part of my thesis. Thank you for your comment. – khorgosh Sep 07 '17 at 15:36
1 Answers
1
- Take a large amount of testing data, enough to cover a meaningful amount of your IV space.
- Apply your classifier to the testing data. Evaluate it, using your favorite error measure.
- Use a regression method to learn how the error of your classifier depends on the input IVs.
Essentially, you would be using a two-step ML procedure, where the second step would be to learn how well the the first step performs.
I recommend that you don't use training data, but test data here. Otherwise you will be very likely to overfit.
Stephan Kolassa
- 95,027
- 13
- 197
- 357