4

I am taking a course in data mining. I am not sure how a non linear SVM when transformed to high dimensional space becomes a linear classification problem. It would be good if someone can provide me an intuition on this.

user3897
  • 517
  • 1
  • 7
  • 13
  • 6
    Check out the third picture: http://stats.stackexchange.com/questions/3947/help-me-understand-support-vector-machines/3954#3954 –  Mar 25 '11 at 14:57

2 Answers2

4

Imagine that you have three classes of points that look like a target with a bullseye: one group in the center, then another circling around that, then another circling around those. You can't take a pair of scissors and cut a straight line to separate any of these groups.

Now imagine that this graph is on a sheet of rubber, and you stick a pin in the middle to hold it down and then pull the corners up, making a funnel shape. You've deformed the points by moving them from 2 dimensions to 3 dimensions. Now, you can make a straight cut with your scissors near the tip of the funnel, totally isolating the first group. Then a second straight snip farther up the funnel separates the second and third groups.

What were curves in a lower dimension can become straight lines in a higher dimension. (In this example, the $z$ is related to the distance from the center.) This isn't the way that SVM actually does its job, but is the way I can see clearly that projecting data to a higher dimension can do nice things.

Wayne
  • 19,981
  • 4
  • 50
  • 99
  • +1 But it isn't necessary for "curves" to become "straight lines." As your image so capably conveys, the images of the curves can still be *curves*, but they may individually be limited to regions that are readily separated by hyperplanes. – whuber Jun 04 '12 at 21:31
2

You should look at the problem the other way arround. SVM algorithms solve Linear classification problems in feature space. Depending on the kernel you use, the boundaries in the original space might not be linear.

Zarbouzou
  • 170
  • 4