Statistical Analysis Stack Exchange
Statistical Analysis Stack Exchange

Questions tagged [svm]

Support Vector Machine refers to "a set of related supervised learning methods that analyze data and recognize patterns, used for classification and regression analysis."

...The standard SVM takes a set of input data and predicts, for each given input, which of two possible classes the input is a member of, which makes the SVM a non-probabilistic binary linear classifier. Given a set of training examples, each marked as belonging to one of two categories, an SVM training algorithm builds a model that assigns new examples into one category or the other. An SVM model is a representation of the examples as points in space, mapped so that the examples of the separate categories are divided by a clear gap that is as wide as possible. New examples are then mapped into that same space and predicted to belong to a category based on which side of the gap they fall on.

--Wikipedia

Visually:

enter image description here

2215 questions
176
votes
8 answers

What is the influence of C in SVMs with linear kernel?

I am currently using an SVM with a linear kernel to classify my data. There is no error on the training set. I tried several values for the parameter $C$ ($10^{-5}, \dots, 10^2$). This did not change the error on the test set. Now I wonder: is…
alfa
  • 2,505
  • 3
  • 15
  • 15
166
votes
5 answers

How to intuitively explain what a kernel is?

Many machine learning classifiers (e.g. support vector machines) allow one to specify a kernel. What would be an intuitive way of explaining what a kernel is? One aspect I have been thinking of is the distinction between linear and non-linear…
hashkey
  • 1,661
  • 3
  • 9
  • 3
128
votes
5 answers

How does a Support Vector Machine (SVM) work?

How does a Support Vector Machine (SVM) work, and what differentiates it from other linear classifiers, such as the Linear Perceptron, Linear Discriminant Analysis, or Logistic Regression? * (* I'm thinking in terms of the underlying motivations for…
tdc
  • 7,289
  • 5
  • 32
  • 62
107
votes
4 answers

How to select kernel for SVM?

When using SVM, we need to select a kernel. I wonder how to select a kernel. Any criteria on kernel selection?
xiaohan2012
  • 6,819
  • 5
  • 18
  • 18
82
votes
1 answer

Help me understand Support Vector Machines

I understand the basics of what a Support Vector Machines' aim is in terms of classifying an input set into several different classes, but what I don't understand is some of the nitty-gritty details. For starters, I'm a bit confused by the use of…
rohanbk
  • 1,187
  • 1
  • 10
  • 10
74
votes
4 answers

What makes the Gaussian kernel so magical for PCA, and also in general?

I was reading about kernel PCA (1, 2, 3) with Gaussian and polynomial kernels. How does the Gaussian kernel separate seemingly any sort of nonlinear data exceptionally well? Please give an intuitive analysis, as well as a mathematically involved…
Simon Kuang
  • 2,051
  • 3
  • 17
  • 18
69
votes
5 answers

Why bother with the dual problem when fitting SVM?

Given the data points $x_1, \ldots, x_n \in \mathbb{R}^d$ and labels $y_1, \ldots, y_n \in \left \{-1, 1 \right\}$, the hard margin SVM primal problem is $$ \text{minimize}_{w, w_0} \quad \frac{1}{2} w^T w $$ $$ \text{s.t.} \quad \forall i: y_i…
blubb
  • 2,458
  • 2
  • 19
  • 28
61
votes
4 answers

Comparing SVM and logistic regression

Can someone please give me some intuition as to when to choose either SVM or LR? I want to understand the intuition behind what is the difference between the optimization criteria of learning the hyperplane of the two, where the respective aims are…
user41799
  • 661
  • 1
  • 6
  • 5
60
votes
5 answers

How does one interpret SVM feature weights?

I am trying to interpret the variable weights given by fitting a linear SVM. (I'm using scikit-learn): from sklearn import svm svm = svm.SVC(kernel='linear') svm.fit(features, labels) svm.coef_ I cannot find anything in the documentation that…
Austin Richardson
  • 928
  • 1
  • 8
  • 10
58
votes
5 answers

Neural networks vs support vector machines: are the second definitely superior?

Many authors of papers I read affirm SVMs is superior technique to face their regression/classification problem, aware that they couldn't get similar results through NNs. Often the comparison states that SVMs, instead of NNs, Have a strong founding…
stackovergio
  • 1,025
  • 1
  • 8
  • 11
58
votes
3 answers

Why do Convolutional Neural Networks not use a Support Vector Machine to classify?

In recent years, Convolutional Neural Networks (CNNs) have become the state-of-the-art for object recognition in computer vision. Typically, a CNN consists of several convolutional layers, followed by two fully-connected layers. An intuition behind…
Karnivaurus
  • 5,909
  • 10
  • 36
  • 52
56
votes
6 answers

What are alternatives of Gradient Descent?

Gradient Descent has a problem of getting stuck in Local Minima. We need to run gradient descent exponential times in order to find global minima. Can anybody tell me about any alternatives of gradient descent as applied in neural network learning,…
Tropa
  • 765
  • 1
  • 7
  • 13
53
votes
2 answers

Linear kernel and non-linear kernel for support vector machine?

When using support vector machine, are there any guidelines on choosing linear kernel vs. nonlinear kernel, like RBF? I once heard that non-linear kernel tends not to perform well once the number of features is large. Are there any references on…
user3269
  • 4,622
  • 8
  • 43
  • 53
47
votes
3 answers

SVM, Overfitting, curse of dimensionality

My dataset is small (120 samples), however the number of features are large varies from (1000-200,000). Although I'm doing feature selection to pick a subset of features, it might still overfit. My first question is, how does SVM handle…
user13420
  • 825
  • 2
  • 9
  • 10
45
votes
6 answers

How to determine the optimal threshold for a classifier and generate ROC curve?

Let say we have a SVM classifier, how do we generate ROC curve? (Like theoretically) (because we are generate TPR and FPR with each of the threshold). And how do we determine the optimal threshold for this SVM classifier?
RockTheStar
  • 11,277
  • 31
  • 63
  • 89
1
2 3
…
99 100