3

I am doing Rainfall-Runoff modeling. I have 4014 inputs and 4014 outputs. I am confused about the of support vectors. Suppose I have a model having 2000 support vectors and I have another model having 3900 support vectors. Both perform well, and the second model has slightly better performance. Which is better? What is the relation between number of support vectors and over fitting?

kjetil b halvorsen
  • 63,378
  • 26
  • 142
  • 467
Vishal mishra
  • 31
  • 1

1 Answers1

1

Look at it this way, in the worst (calling this worse is going ahead of the game, but an explanation as to why this is the worst case would soon follow) case, all your input points could turn out to be support vectors. That means that your separating hyperplane will have a lot of twists and turns. Imagine that you went to an infinite dimensional space (using a kernel), effectively with infinite parameters. If all your points become support vectors, it would mean an extremely high number of "effective" parameters, resulting in horrible generalization and thus, gross overfitting.

statBeginner
  • 1,251
  • 2
  • 17
  • 22