1

Objective

To confirm if the understanding is correct regarding the reason why Lagrangian Dual is used in SVM.

Background

While Machine Learning, gradient descent is used at regression, logistic regression, back propagation, etc. However for SVM, suddenly Lagrangian Dual pops up. Trying to understand why.

enter image description here

It looks to me gradient descent is possible to solve "primary" issue of SVM as in the left side of the image below (please correct if it is wrong). However never saw any articles or books which mention using gradient descent. Instead, always Lagrangian Dual and KKT conditions pop up out of blue and do not explain well why.

enter image description here

Question

I suppose using the dual reduces the dimensions , e.g (x1, x2) to λ in the image, and results in a formula that produces definite solutions.

  • Is it the only reason to use Lagrangian Dual, or are there any other reasons?
  • Is there any reason why Gradient Descent cannot be used?
mon
  • 685
  • 5
  • 16
  • 2
    http://olivier.chapelle.cc/primal/ – seanv507 Jan 10 '17 at 09:23
  • 1
    I also once answered something on the subject of primal SVM mentioning Chapelle as well: http://stats.stackexchange.com/questions/215524/is-gradient-descent-possible-for-kernelized-svms-if-so-why-do-people-use-quadr/230579#230579. Primal Sub-Gradient Descent is possible, and the dual doesn't guarantee approximate solutions. – Firebug Jan 10 '17 at 12:21
  • 1
    @seanv507, thanks a lot for the references, especially the link to Chapelle's paper. – mon Jan 10 '17 at 21:14
  • 2
    @Firebug, thanks for the reference and pointer to Chapelle's article. – mon Jan 10 '17 at 21:15

0 Answers0