8

I did some tutorials and read few articles but still have a problem with SVM, exactly with SVR. I'm doing analysis in R and I use e1071 library with "svm" function. Into that function I use my multivariable equation, so svm works since now like SVR.

My results:

(general: cost=1,gamma=0.1666)
-epsilon=0.1(61 SV-supported vectors) - RMSE = 4.1(on unseen data)
-epsilon=1(10 SV) - RMSE = 19(on unseen data)
-epsilon=1.3(7 SV) - RMSE = 25(on unseen data)

When epsilon is increasing I understand that we should have actually more supported vectors as we can see on the picture below:

SVR

In our case as we can see the bigger epsilon the less supported vectors. I don't know why this happens. I would be glad if someone can explain it to me.

Ferdi
  • 4,882
  • 7
  • 42
  • 62
boplight
  • 103
  • 1
  • 1
  • 5

1 Answers1

10

You have it backwards.

Traditional $\epsilon$-SVR works with the epsilon-insensitive hinge loss. The value of $\epsilon$ defines a margin of tolerance where no penalty is given to errors.

Remember the support vectors are the instances across the margin, i.e. the samples being penalized, which slack variables are non-zero.

The larger $\epsilon$ is, the larger errors you admit in your solution. By contrast, if $\epsilon \rightarrow 0_+$, every error is penalized: you end with many (tending to the total number of instances) support vectors to sustain that.

Firebug
  • 15,262
  • 5
  • 60
  • 127