1

After reading many posts, I thought of asking: Why should a SVM be biased towards majority class like other classifiers, since an SVM never used the whole data of the training data set—it only uses the support vectors to determine the best hyperplane, maximizing the margins between two classes.

For example, out of 1000 records, having 900 vs 100 class observations for a binary classification problem, then SVM might only use 50 support vectors and leave others. In this case, will it be biased towards majority class? That is, will my ROC curve not show good results?

Posting few links stating this issue with Svm, it is suggesting to use change cost of misclassification for different classes, like any other classifier that uses the entire datasets:

https://www.google.co.in/url?sa=t&source=web&rct=j&url=http://www.cs.colostate.edu/~cs545/fall13/dokuwiki/lib/exe/fetch.php%3Fmedia%3Dwiki%253A05_svm_unbalanced.pdf&ved=0ahUKEwjLt-CE8uXUAhWLKo8KHTNKCakQFgiBATAG&usg=AFQjCNFtw1url6ws4yxoaZMUzpoWpoFx5w

SVM for unbalanced data

http://www.kdnuggets.com/2016/04/unbalanced-classes-svm-random-forests-python.html/3

Argho Chatterjee
  • 11
  • 1
  • 4
  • 2
    You mentioned that you've read a number of posts. Could you link to some of those in your post, so that other people that may stumble upon this question in the future can see where your train of thought came from? – Mark White Jun 30 '17 at 13:13
  • 2
    Sure I have added a few links stating to use class weight parameters, i. e. Cost sensitive learning to improve SVM performance on imbalanced datasets – Argho Chatterjee Jun 30 '17 at 15:43

1 Answers1

1

Although the SVM is using only support vectors for prediciton, it uses all the data during the training process. You can think about the training process as to looking for parameters describing your decision boundary, such as they will minimize some kind of average misclassification error function. If you have 9 times samples from class A as from class B, they will add 9 times as much error. So the decision boundary will be pushed towards the minority class.

rep_ho
  • 6,036
  • 1
  • 22
  • 44