2

I'm not clear about kernel.

  1. How could I construct my own kernel that is valid? Is the only method the Mercer Theorem (positive semi-definite)? I mean if I know $K$ is a valid kernel, do I know that $aK+bK$, $\mathrm{polynomial}(K)$, $\exp(K)$, etc., are valid as well?
  2. If $K$ is a 2-dimension kernel, how could I expand (create) $K$ to 3-D?
  3. Given a kernel, say $K(x,y)=(x^Ty+1)^3$, how could I find a $\varphi$ that make this $K = \varphi^T \varphi$ hold?
Danica
  • 21,852
  • 1
  • 59
  • 115
Freya Ren
  • 163
  • 1
  • 5

1 Answers1

2

  1. Proving that the kernel is positive definite is sufficient. It's also true that various operations on kernels preserve kernel-ness; I recently catalogued some here, but many machine learning textbooks contain lists of such things.

  2. One way is to use a product kernel: let $K_1$ be the kernel on the first two dimensions, $K_3$ one on the third dimension, and multiply them. There are also of course many other methods, depending on the specific kernel.

  3. In general, this is quite difficult. In that specific case, you can expand out the polynomial and separate factors of various powers of each $x$ or $y$.

Community
  • 1
Danica
  • 21,852
  • 1
  • 59
  • 115
  • and how to separate factors of various powers of each or ? – Anna Leonenko Mar 06 '21 at 19:50
  • @AnnaLeonenko Let's say in 1d: $$(x y + 1)^3 = x^3 y^3 + 3 x^2 y^2 + 3 x y + 1 = \begin{bmatrix}x^3 \\ \sqrt{3} x^2 \\ \sqrt{3} x \\ 1\end{bmatrix}^T \begin{bmatrix}y^3 \\ \sqrt{3} y^2 \\ \sqrt{3} y \\ 1\end{bmatrix},$$ so $\phi(x) = (x^3, \sqrt 3 x^2, \sqrt 3 x, 1)$ works. In higher dimensions the same will hold, there are just more terms. [`sklearn.preprocessing.PolynomialFeatures`](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.PolynomialFeatures.html) does this for you. – Danica Mar 08 '21 at 22:20
  • Thanks @Danica! – Anna Leonenko Mar 16 '21 at 13:21