21

I need to classify URLs into categories. Say I have 15 categories that I'm planning to zero down every URL to.

Is a 15-way classifier better? Where I have 15 labels and generate features for each data point.

Or building 15 binary classifiers, say: Movie or Non-Movie, and use the numbers I get from these classifications to build a ranker, to pick the best category, going to be better?

Macro
  • 40,561
  • 8
  • 143
  • 148
madCode
  • 736
  • 1
  • 5
  • 14

3 Answers3

12

First of all, you must ask yourself if your problem is multilabel (i.e. a single URL can belong to several classes) or not (i.e. a single URL can belong to only one class).

If the former, go with a battery of binary classifiers, because this is a default way of doing multilabel problems.

If the latter, the answer depends on a combination of how does your data look, what is the aim of your analysis and what method are you using -- probably you should just try both and select best.
Only note that some methods (like SVM) can't actually do multiclass classification because of how they are defined and thus internally use a battery of binary classifiers.

  • my problem statement is considering the former assumption @mbq. I know there are multilabels. and yes, like you've said I have decided to go for 15 binary classifiers but again, I need to rank them to pick one best category. So, I'm going to try performing another top level classification using the numbers I obtained from the battery of binary classifiers. Do you see any problem? – madCode Jun 19 '12 at 15:04
  • SVMs can perform multiclass classification. The method is very similar to softmax regression (see "On the algorithmic implementation of multiclass kernel-based vector machines"). – user1149913 Aug 08 '12 at 13:05
4

This will depend on how your data is dispersed. There is a beautiful example that was given recently to a similar question where the OP wanted to know if a single linear discriminant function would be a better classifier for deciding population A vs B or C or one based on mutliple linear discriminant functions that separate A ,B and C. Some one gave a very nice colored scatterplot to show how using two discriminants would be better than one in that case. I will try to link to it.

Michael R. Chernick
  • 39,640
  • 28
  • 74
  • 143
  • Hang on. I am having trouble finding it but I will keep looking. – Michael R. Chernick Jun 18 '12 at 18:48
  • Sorry for not being able to find the link. Imagine a cloud of one color to the left, another in the middle and a third to the right. Two linear discriminant line would do a good job of separating the middle group from the ones to the left and right but no single line would do well at all. The picture would be worth more than all these words. – Michael R. Chernick Jun 18 '12 at 20:38
  • 1
    @MichaelChernick Is this the [link](http://stats.stackexchange.com/questions/30075/multi-class-lda-vs-2-class-lda) you are looking for? –  Jun 18 '12 at 21:00
  • I think I understand what you are saying: http://bit.ly/M1NydS - the picture you defined I came across in this presentation. 4 way or 3 way classification..could be direct. But.. I'm wondering if the precision/recall would be compromised if do 15 way classification, Dr. Chernick. – madCode Jun 18 '12 at 21:10
  • @Procrastinator Thank you for finding that. i was having so much trouble locating it and I was sopemnding a lot of time looking! It was a recent post so I though tit would be easy to find. – Michael R. Chernick Jun 19 '12 at 00:37
  • @madCode Let me give your question some thought and I will try to give you a good answer. – Michael R. Chernick Jun 19 '12 at 00:37
  • @madCode I think slicing up the space 15 ways may be a bit too much. – Michael R. Chernick Jun 20 '12 at 03:38
  • @MichaelChernick hmmm.. I'm gonna do the 15 Binary ---> find a way to Rank..and see how the precision/recall is! Will keep you posted. – madCode Jun 20 '12 at 16:45
1

Some methods deal well with multiclass, Random Forests, MLPs for example.

If you don't want to go that way, then it is possible that ECOC may well out perform 1-vs-All for your problem, only testing will tell.

image_doctor
  • 750
  • 5
  • 9