1

I have been going through the literature for SVM regression and I find that majority of the focus is on selecting a model using grid search. This is a completely data driven approach.

I would like to see a more knowledge driven approach in selecting the hyperparameters epsilon, gamma and C. Please refer me to a good paper or article highlighting the workflow for selection of SVM parameters by a knowledge driven approach based on understanding of the various features and data points.

Ankit Bansal
  • 165
  • 1
  • 2
  • 11

1 Answers1

1

I would say grid search is an engineering approach instead of mathematical approach. In addition, the definition of "more intuitive and realistic approach" is not clear to me.

For any hyper parameter tuning in machine learning (including SVM gamma and C), we need to consider the bias variance tradeoff. For example, C is the regularization parameter, we need to know making it bigger or smaller will fix overfitting or underfitting (and currently where we are).

Many of my answers covered this basic idea of checking overfitting or underfitting. Do not know if you think these are "intuitive and realistic".

How to know if a learning curve from SVM model suffers from bias or variance?

How to chose the order for polynomial regression?

Community
  • 1
Haitao Du
  • 32,885
  • 17
  • 118
  • 213
  • Suppose I have a data set with twenty features like fuel injection rate, temperature, rpm etc about an internal combustion engine describing the failure points. Rather than performing a grid search to determine the parameters of SVM regression, I would like to have an approach that lets me make a better selection based on the the datapoint of fuel injection rates that I believe can be outside the epsilon range, or select C or gamma defining the datapoints that act as support vectors based on visual/realistic judgement, rather than making a decision based on the lowest mean error. – Ankit Bansal Mar 07 '17 at 17:58
  • I'm basically looking for a knowledge driven application of SVM. I'm editing the question to make it more lucid – Ankit Bansal Mar 08 '17 at 07:25