0

During the SVM formulation, the 2 hyperplanes is given by the equations:

wᵀx + b = 1 ---------(1)

wᵀx + b = -1 ---------(2)

Now, the margin between these 2 hyperplanes is given by: 2/||w||

However, I'm not able to derive the margin 2/||w|| from the equations 1 and 2 geometrically.

I've tried this:

Consider a point 'p' on plane (1). Then, the distance of that point from plane (2) will be:

(wᵀp)/||w|| ---------(3)

which will also be the distance between the 2 planes.

Since p lies on plane (1), we have:

p = (1-b) * inverse(wᵀ) ----------(4)

On substituting the value of p in equation (3), we have:

wᵀ/||w|| * (1-b) * inverse(wᵀ)

= (1-b)/||w|| ---------------------(5)

Can anyone please tell me how to eliminate 'b' from the last equation, such that we can arrive at the distance 2/||w|| ?

Saurabh Verma
  • 171
  • 7

2 Answers2

1

These are parallel hyperplanes, with normal vectors $\mathbf{w}$. Pick a point on plane (1), call it as $\mathbf{p}_1$, and let the corresponding point be $\mathbf{p}_2$ on plane (2). We can reach from $\mathbf{p}_1$ to $\mathbf{p}_2$ by moving in the direction of $\mathbf{w}$ (or $-\mathbf{w}$), i.e. we have the following relation: $\mathbf{p}_1+\mathbf{w}t=\mathbf{p_2}$ (3). And, we want to find $d=||\mathbf{p}_1-\mathbf{p}_2||=|t|||\mathbf{w}||$ which is indeed the distance between the planes.

Subtract equation (2) from (1), and we have: $\mathbf{w}^T(\mathbf{p}_1-\mathbf{p}_2)=2$. Substitute (3) here: $-||\mathbf{w}||^2t=2$, which means $t=-2/{||\mathbf{w}||^2}$. Then, the distance is $d=|t|||\mathbf{w}||=2/||\mathbf{w}||$.

Comment on your way: there is no "inverse(wᵀ)" firstly because $\mathbf{w}$ is not a square matrix.

gunes
  • 49,700
  • 3
  • 39
  • 75
0

What is the geometric margin? Geometric margin is the shortest distance between points in the positive examples and points in the negative examples. Now, the points that have the shortest distance as required above can have functional margin greater than equal to 1. However, let us consider the extreme case when they are closest to the hyperplane that is, the functional margin for the shortest points are exactly equal to 1. Let $x_+$ be the point on the positive example be a point such that $w^Tx_+ + w_0 = 1$ and $x_-$ be the point on the negative example be a point such that $w^Tx_- + w_0 = -1$. Now, the distance between $x_+$ and $x_-$ will be the shortest when $x_+ - x_-$ is perpendicular to the hyperplane.

Now, with all the above information we will try to find $\|x_+ - x_-\|_2$ which is the geometric margin. $$w^Tx_+ + w_0 = 1$$ $$w^Tx_- + w_0 = -1$$ $$w^T(x_+ - x_-) = 2$$ $$|w^T(x_+ - x_-)| = 2$$ $$\|w\|_2\|x_+ - x_-\|_2 = 2$$ $$\|x_+ - x_-\|_2 = \frac{2}{\|w\|_2}$$

Referred from: Given a set of points in two dimensional space, how can one design decision function for SVM?

Raveendar V
  • 1
  • 1