0

I am using sklearn.SGDClassifier to train my SVM model with loss='hinge'. My questions are:

  1. Is there a way to get support vectors number by having this SGD model? I found this online but it is not helpful: How to identify support vectors in SGD svm?

  2. Does anyone can provide the formal relation between C and alpha parameter in sklearn.SGDClassfier as an SVM as well, with references if possible? Apparently, SVM's C is not an attribute in SGDClassifier. These two links are providing different relations and I am confused:

https://stackoverflow.com/questions/34556476/regularization-parameter-and-iteration-of-sgdclassifier-in-scikit-learn

How does alpha relate to C in Scikit-Learn's SGDClassifier?

Thank you very much for your help. :)

Aaron Lieu
  • 3
  • 4

1 Answers1

0

Answering your second question.

C parameter in SVC multiplies the loss term. Increasing C will increase the penalty for misclassification. SVC uses liblinear library. You can find more information about the loss function and the C parameter on liblinear documentation: https://www.csie.ntu.edu.tw/~cjlin/papers/liblinear.pdf

alpha parameter in SGD multiplies the regularization term. There is no C parameter in SGD because you can use SGD with different loss functions and regularization methods.

In their effect, C and alpha are inversely related: increasing C in SVC and decreasing alpha in SGD will lead to overfitting.

ozen
  • 16
  • I understood that C and alpha are inversely related, but, do you have the **formal relation** on this so that I can have some calculations on it? – Aaron Lieu Feb 28 '20 at 15:15
  • I think the answer you are looking for does not exist. SVM and SGD are two different methods. The optimization problem SVC solves is given in Eq.1. in the link I provided. SGD iteratively optimizes x + where alpha, penalty, and loss are parameters of SGDClassifier. So there are two different formulas. When you use hinge loss and l2 penalty, the formulas are almost the same, except one multiplies the loss with C and the other multiples the penalty with alpha. – ozen Feb 29 '20 at 16:44