4

If I have more than 3 features in a dataset, then I can't visualize them to see if my classes are scattered in a non linear fashion. So how do I know when is the right way to use linear model with non-linear(polynomial) basis function in logistic regression or Support Vector Machines?. I just don't want to use a kernel function in SVM without knowing why that should be used.

Is there a way to know when to use Kernel functions of non linear basis functions in a linear model by looking at the data?

Sean Easter
  • 8,359
  • 2
  • 29
  • 58
Baktaawar
  • 825
  • 2
  • 11
  • 18
  • 3
    Possible duplicate of [Linear kernel and non-linear kernel for support vector machine?](http://stats.stackexchange.com/questions/73032/linear-kernel-and-non-linear-kernel-for-support-vector-machine) – Marc Claesen Nov 13 '15 at 20:54
  • 5
    I'm voting to leave open, as the OP's focus seems not on which to use, but on whether one can somehow tell which to use *without* running a linear kernel SVM. (It might well be that the answer is, "No, you pretty much have to run an SVM or another linear classifier," but the questions strike me as distinct.) – Sean Easter Nov 13 '15 at 21:24
  • By looking at the data, no. You should start with the linear model and plot learning curves (error as a function of the training set size). Plot both training and testing errors. If the both errors converge to some plateau and in absolute value are high, then you are in high bias setting - you need more powerful models, e.g. non-linear kernel SVM. Check out Andrew Ng's lecture about Learning Curves (https://class.coursera.org/ml-005/lecture/64). – Vladislavs Dovgalecs Nov 13 '15 at 22:01
  • For completeness, if the gap between two curves is large and the training error is very low, you are in high variance or overfitting setting. Here some regularization and/or more data can help. – Vladislavs Dovgalecs Nov 13 '15 at 22:03

0 Answers0