Questions tagged [max-margin]
13 questions
21
votes
1 answer
Interpreting distance from hyperplane in SVM
I have a few doubts in understanding SVMs intuitively. Assume we have trained a SVM model for classification using some standard tool like SVMLight or LibSVM.
When we use this model for prediction on test data, the model generates a file having…
Amit
- 743
- 2
- 6
- 16
10
votes
1 answer
What's the relationship between an SVM and hinge loss?
My colleague and I are trying to wrap our heads around the difference between logistic regression and an SVM. Clearly they are optimizing different objective functions. Is an SVM as simple as saying it's a discriminative classifier that simply…
Simon
- 268
- 2
- 9
5
votes
2 answers
Is there a way to remove individual trees from a forest in the randomForest package in R?
I am trying to implement the ideas in this paper: http://www.sciencedirect.com/science/article/pii/S0925231212003396.
This requires me to be able to remove individual trees from the forest and reclassify my training data for each removal. I've been…
Spy_Lord
- 307
- 3
- 9
3
votes
1 answer
Max-margin clustering with size constraint
Given a dataset $D$ and a distance measure, I want to split the dataset into two disjoint subsets $X, Y$ of a specified size (say 80% and 20% of the original size), so that the minimum distance of all pairs $(x, y)$ with $x \in X$ and $y \in Y$ is…
etarion
- 131
- 2
2
votes
0 answers
Linear SVM decision boundary after a linear transformation of data
Let $w$ be the decision boundary of a linear SVM trained on the dataset $D=\{(x_i, y_i)_{i=1}^N\}$. Suppose we apply a linear transformation A to examples $x_i$s and obtain a new dataset $D'=\{(z_i, y_i)_{i=1}^N\}$ where $z_i = Ax_i$. And let $w_2$…
emrea
- 1,021
- 5
- 12
2
votes
1 answer
Relationship between L1 penalty and margin in SVM
Expanding on "Why aren't there there two regularization terms in SVC?" and "Meaning of penalty and loss in LinearSVM":
It appears that LinearSVC in Python also supports $l_1$ regularisation ("penalty") of the class boundary vector $\mathbf{w}$, in…
Igor F.
- 6,004
- 1
- 16
- 41
1
vote
0 answers
Show that solution for the maximum margin hyperplane is unchanged when w.x + b = (+/-) 1 is replaced by arbitrary constant $\gamma$?
How to show that solution for the maximum margin hyperplane for hard-margin SVM is unchanged when w.x + b = (+/-) 1 is replaced by arbitrary constant $\gamma$?
In the derivation for the SVM, we generally assume that the margin boundaries are given…
Tuhin Dutta
- 21
- 1
1
vote
1 answer
IS optimization unnecessary in SVM?
According to here, Now knowing the $a_i$ we can find the weights $w$ for the maximal margin separating hyperplane:
\begin{align*}
w = \sum_{i=1}^{l} a_i y_i x_i
\end{align*}
I cannot understand what this says. I have trouble in how to choose $a_i$.…
yoyo
- 979
- 1
- 6
- 9
1
vote
1 answer
CRF Training: Max-margin vs max-likelihood
I'm trying to use PyStruct's CRF implementation. In its user guide, it says the following:
I call these models Conditional Random Fields (CRFs), but this a
slight abuse of notation, as PyStruct actually implements perceptron
and max-margin…
Veech
- 190
- 1
- 6
0
votes
0 answers
Given SVM margin, calculate the optimal parameters
So given the margin as in $\frac{1}{||\theta||}$, calculate $\theta$. I was given this question as part of an exercise, but it doesn't seem possible as $\frac{1}{||\theta||}$ is some sort of scalar so I don't know how you can derive $\theta$, which…
user8714896
- 640
- 2
- 12
0
votes
0 answers
Is Symmetric hinge loss affected by outliers
Hinge loss can be affected by outliers so I read, I was wondering if the same applies to symmetric hinge loss defined by $max(0,|1−w^{T}x_i|)$ for clustering
Edward Chome
- 1
- 1
0
votes
0 answers
Does gradient descent for linear regression selects the minimal norm solution?
I was told that Gradient Descent finds the weights of smallest norm.
This is what I understood in the linear regression setting:
$f_w(x)=w^\top x$ are the linear functions $ \mathbb{R}^n \rightarrow \mathbb{R} $.
$ \mathcal{D}=\{(x_i,y_i) \in…
rod
- 101
- 1
0
votes
1 answer
How to calculate the margin in SVM light?
I'm using Support Vector Machine in a project. The library chosen is SVM light of Joachims: http://svmlight.joachims.org/
I have the need to calculate the margin. Namely, given a training set of data I have to calculate the margin of the better…
Umbert
- 133
- 7