0

With feature scaling we just change representation of the data. This can make our model run faster but how this can improve accuracy? It is the same data after all.

When I train my SVM without feature scaling I get an accuracy of %61, after MinMax scaling from sklearn accuracy increases to %96. How is this possible even we work with same data and use the same model?

J.Smith
  • 103
  • 1
  • 2
    Some models (not all) are sensitive to scale, because they unjustly give more weight to features with bigger scales. Scaling here improves the model because it puts all features on the same scale, giving the model a better chance of finding the right patterns. – user2974951 Aug 20 '19 at 10:35
  • 1
    A reason it can be possible is because you’re almost never finding a global optimal in machine learning, and for some models having a feature that dominates causes the learning algorithm to get stuck in a bad local optimal solution. – Joe Aug 20 '19 at 11:46

1 Answers1

3

SVM is an algorithm that is based on distances since it tries to maximize the margin between the support vectors.

If one of the features would have much larger values than another it would dominate the distance results and the other feature would have a lesser effect on the result even if it changes significantly.

By scaling the features to the same range, the algorithm would be sensitive to all of them and not biased to the features with the greater magnitude.

Romid
  • 146
  • 1