32

What is the difference between a one-vs-all and a one-vs-one SVM classifier?

Does the one-vs-all mean one classifier to classify all types / categories of the new image and one-vs-one mean each type / category of new image classify with different classifier (each category is handled by special classifier)?

For example, if the new image to be classified into circle, rectangle, triangle, etc.

gung - Reinstate Monica
  • 132,789
  • 81
  • 357
  • 650
user3378327
  • 951
  • 2
  • 8
  • 11

1 Answers1

44

The difference is the number of classifiers you have to learn, which strongly correlates with the decision boundary they create.

Assume you have $N$ different classes. One vs all will train one classifier per class in total $N$ classifiers. For class $i$ it will assume $i$-labels as positive and the rest as negative. This often leads to imbalanced datasets meaning generic SVM might not work, but still there are some workarounds.

In one vs one you have to train a separate classifier for each different pair of labels. This leads to $\frac{N(N-1)}{2}$ classifiers. This is much less sensitive to the problems of imbalanced datasets but is much more computationally expensive.

Gnattuha
  • 1,004
  • 6
  • 10
  • Please, did you mean *i-labels as positive* OR *i-th label as positive* ? – PeterB May 13 '17 at 00:10
  • labels corresponding to the class i as positive. – Gnattuha May 14 '17 at 21:05
  • @Gnattuha - What do you mean by imbalanced datasets? Thanks in advance. – saurabheights Mar 21 '18 at 17:34
  • 1
    I read here - https://en.wikipedia.org/wiki/Multiclass_classification#Transformation_to_binary - "Although this strategy is popular, it is a heuristic that suffers from several problems. Firstly, the scale of the confidence values may differ between the binary classifiers. Second, even if the class distribution is balanced in the training set, the binary classification learners see unbalanced distributions because typically the set of negatives they see is much larger than the set of positives". Still how does that imbalancing affect the accuracy? – saurabheights Mar 21 '18 at 17:44