2

The linear SVM function can be expressed as:

$$h(x)=w^{T}x+w_{0}$$

where $x=({{x}}_{1},\dots,{{x}}_{N})^{T}$ is one observation and $w=({{w}}_{1},\dots,{{w}}_{N})^{T}$ are the coefficients.

We can do the calculation of the dot product in the following way:

$$w^{T}x=x_1w_1+\dots+x_Nw_N$$

Now with the kernel trick, the function $h$ is expressed as:

$$h(x)=w^{T}\phi (x)+w_{0}$$

And my question is : for a given observation $x$, how the calculation is done ? since, $\phi$ is always expressed as:

$$K(x_{i},x_{j})=\phi (x_{i})^{T}\cdot \phi (x_{j})$$

With some example of $K$

  • Polynomial kernel : $K(x_{i},x_{j})=(x_{i}^{T}\cdot x_{j}+1)^{d} $
  • Gaussian kernel: $ {\displaystyle K(\mathbf {x} ,\mathbf {y} )=\exp \left(-{\frac {\|\mathbf {x} -\mathbf {y} \|^{2}}{2\sigma ^{2}}}\right)}$

PS: with the dual form, we can express $h$ as:

$$h(x)=\sum _{{k=1}}^{p}\alpha _{k}^{*}l_{k}K(x_{k},x)+w_{0}$$

But I try to understand how to write the function $\phi$ in an explicit way.

Sycorax
  • 76,417
  • 20
  • 189
  • 313
John Smith
  • 158
  • 3
  • 17
  • 1
    The expression you're asking about ($h(x) = w^T \phi(x) + w_0$) does not use the kernel trick. The point of the kernel trick is to avoid explicit computation of the feature map $\phi$, which may map to a very high (or possibly infinite) dimensional space. The 'dual form' expression you wrote for $h(x)$ is how the kernelized version should look. – user20160 Dec 29 '21 at 14:34
  • @user20160 This seems like an answer to the question. Would you care to write it up? – Sycorax Dec 29 '21 at 17:17
  • Thank you for your comment, I found [this post](https://stats.stackexchange.com/questions/463255/kernels-in-svm-primal-form?rq=1) useful to understand that apply $\phi$ is not the kernel trick. But then I don't know how to say it exactly, maybe we should say: apply $\phi$ to project x to a feature map. Which one ? if it is polynomial, do we say polynomial feature map ? or feature map with polynomial kernel ? for the latter, we use "kernel"... – John Smith Dec 30 '21 at 15:40
  • To express explicit $\phi$ for a gaussian kernel (notice the word kernel is used), this [post](https://stats.stackexchange.com/questions/69759/feature-map-for-the-gaussian-kernel?rq=1) gives the answer. And for a polynomial kernel, the [wikipedia article](https://en.wikipedia.org/wiki/Polynomial_kernel) gives the answer. – John Smith Dec 30 '21 at 15:46

0 Answers0