0

I found there are two forms of RBF function.

  1. these is a coefficient before $\exp$ $$ k_{f}\left(x_{i}, x_{j}\right)=\sigma^{2} \exp \left(-\frac{1}{2 \ell^{2}} \sum_{j=1}^{q}\left(x_{i, j}-x_{k, j}\right)^{2}\right) $$ which can be found in :

    Kernels in Gaussian Processes

    https://nipunbatra.github.io/blog/ml/2020/06/26/gp-understand.html#:~:text=The%20most%20commonly%20used%20kernel,exponential%20kernel%20%E2%80%93%20all%20are%20equivalent.&text=It%20has%20two%20parameters%2C%20described,2%20and%20the%20lengthscale%20l.&text=rbf.

  2. these is no coefficient before $\exp$ $$ K\left(\mathbf{x}, \mathbf{x}^{\prime}\right)=\exp \left(-\frac{\left\|\mathbf{x} \quad \mathbf{x}^{\prime}\right\|^{2}}{2 \sigma^{2}}\right) $$ which can be found in wikipedia and scikit-learn:

    https://en.wikipedia.org/wiki/Radial_basis_function_kernel

    https://scikit-learn.org/stable/modules/gaussian_process.html#gp-kernels

Now the question is what’s the difference between the two forms? which is True ?

Joey Gao
  • 99
  • 2

1 Answers1

0

They are equivalent. The point of basis functions is to be able to use weighted linear combinations of them to approximate other functions, and the weighted linear combinations of these two give exactly the same set of approximations.

Thomas Lumley
  • 21,784
  • 1
  • 22
  • 73
  • The coefficient $\sigma$ in first equation is weight in `weighted linear combinations`. Am I right? But in the second link, it said `It has two parameters, described as the variance $\sigma$ and the lengthscale $l$`. why we donot need the variance $\sigma$ in GaussianProcess in scikit-learn's implementation? – Joey Gao Jan 27 '21 at 04:24
  • They're equivalent for some applications but not all. E.g. when specifying the covariance function of a Gaussian process, the scale factor $\sigma^2$ in the first form is very important – user20160 Jan 27 '21 at 14:36
  • Everyone who uses the second form to define the covariance function will put in a scale factor, though. – Thomas Lumley Jan 27 '21 at 20:43