3

By the reproducing kernel Hilbert space (RKHS) property, given a P.S.D. kernel function $\kappa:X\times X \rightarrow \mathbb R$, there exists a Hilbert space $H$ and a map $\phi:X\rightarrow H$ such that $$ \kappa(x,y) = \langle \phi(x), \phi(y) \rangle_H. $$

Define $$ \kappa(x,y) = \exp\left(\frac{-\|x-y\|^2}{2\sigma^2}\right) $$ and denote by $H$ the corresponding RKHS.

The definition of the kernel implies that $H$ is defined over field $\mathbb R$. By the properties of the inner product, it should be that for any $\alpha \in \mathbb R$, $$ \langle \alpha\phi(x), \phi(y) \rangle_H = \alpha\langle \phi(x), \phi(y) \rangle_H. $$ This means that we can choose vectors $\phi(x),\phi(y) \in H$ to achieve any real value as a result of $\kappa(x,y)$. However, the image of $\kappa$ is $(0,1]$, so I guess that something must be off in my reasoning.

What is it?

cangrejo
  • 2,121
  • 13
  • 22

1 Answers1

2

It is true that $\kappa(x, y) = \langle \phi(x), \phi(y) \rangle_H$. So for any $x$, $y$, with this choice of $\kappa$, $\kappa(x, y) \in (0, 1]$.

It is also true that, say, $-\phi(x) \in H$, and $\langle - \phi(x), \phi(y) \rangle_H = - k(x, y) \in [-1, 0)$.

It is not true that there is necessarily some $x'$ such that $-\phi(x) = \phi(x')$. The feature mapping $\phi$ doesn't cover all of $H$.

Danica
  • 21,852
  • 1
  • 59
  • 115
  • 1
    Wonderfully simple, now that I think of it. An immediate insight, which I'd never stopped to think about before now, is that this kernel is defined only for the points on the unit sphere in $H$. – cangrejo Nov 01 '19 at 09:55