1

I have read in the MATLAB documentation, that the number of Support Vectors is connected to the value of the soft-margin parameter C - why is that, and how can we see that with regards to the objective function for SVM?

Edit: I understand the overall relevance of C (as discussed in the comments), but have not previously encountered statements regarding C and the number of support vectors. I don't think that the linked post answers my question.

Pugl
  • 951
  • 1
  • 16
  • 40
  • 1
    what is your understanding of the C parameter? – Krrr Sep 05 '17 at 09:09
  • That it (roughly) determines how much we fit the classifier to the particular dataset at hand: Smaller C values will allow for more missclassifications on the particular dataset. I would assume that this means fewer support vectors (since the decision boundary is not as specific as for larger C), but apparently, the contrary is the case – Pugl Sep 05 '17 at 09:47
  • This is what [MATLAB documentation says about the parameter C in svmtrain](https://www.mathworks.com/help/stats/svmtrain.html). I do not see any mention of influence on number of support vectors. Perhaps you are referring to some other documentation? – Krrr Sep 05 '17 at 10:00
  • It says the contrary: "To decrease the number of support vectors, set BoxConstraint to a large value", also it would be nice to see how this follows from the formulation of the objective function – Pugl Sep 05 '17 at 10:05
  • Here it says something about the connection: https://ch.mathworks.com/help/stats/fitcsvm.html#bt8v_z4-1 But I am generally interested in understanding this, not only specifically regarding MATLAB:) – Pugl Sep 05 '17 at 11:37
  • The margin of the induced classifier depends on the C parameter. So [if you increase C then the margin decreases](https://stats.stackexchange.com/questions/31066/what-is-the-influence-of-c-in-svms-with-linear-kernel). This, I imagine in ideal cases, also reduces number of support vectors. But I do not see a reason for this to be the case for all data sets. – Krrr Sep 05 '17 at 12:49
  • and *why* does this reduce the number of support vectors..? – Pugl Sep 05 '17 at 13:20
  • Please have a look at the discussion I linked to above and think what happens if you have well separated data. – Krrr Sep 05 '17 at 15:29
  • That is not really a duplicate, since I understand the role of C in general, but would like to understand its connection to the number of support vectors – Pugl Sep 06 '17 at 22:09

0 Answers0