0

I know that this question can sound somewhat trivial, but I'll ask it nevertheless. When trying to implement the function that computes the gaussian kernel over a set of indexed vectors $\textbf{x}_k$, the symmetric Matrix that gives us back the kernel is defined by $$ K(\textbf{x}_i,\textbf{x}_j) = \exp\left(\frac{||\textbf{x}_i - \textbf{x}_j||}{2 \sigma^2} \right) $$ The problem is, how do I came up with the right choose of $\sigma^2$? I wrote a MATLAB script that implements the above formula as

$$ K(\textbf{x}_i,\textbf{x}_j) = \exp\left(\frac{||\textbf{x}_i - \textbf{x}_j||}{2 \sigma(\textbf{x}_i) \sigma(\textbf{x}_j)} \right) $$ Where $\sigma(\cdot)$ represents obviously the standard deviation computed for each vector. The first problem that comes in my mind is that this cannot be adapted to work with 1-dimensional measurements. Any suggestion about how to tackle this problem? I have no experience in dealing with kernel functions, so any help would be greatly appreciated. I include matlab code here:

function ret = CompGaussKernel(X)

[l,N] = size(X);
% Computation of the Kernel values K(xi,xj)
ret = zeros(N,N);

for i = 1 : N
    ret(:,i)= Kernel(X',X(:,i)');
end

function k = Kernel(u,v)
[r1 c1] = size(u);
[r2 c2] = size(v);
k = zeros(r1,1);
for j = 1 : r1
    k(j) = exp(-(u(j,:)-v)*(u(j,:)-v)'/(2*std(u(j,:))*std(v)));
end
end

end
james42
  • 209
  • 5
  • 14
  • 2
    The method you've implemented is not typically how this is done. Most commonly, people will try, say, ten values of $\sigma$ and use the one that does best on out-of-sample data. – Sycorax Dec 14 '16 at 17:26
  • Thanks a lot! And what about the various values of $\sigma$? Is it usual to assume $0\leq \sigma \leq 1$? – james42 Dec 14 '16 at 22:49
  • A common strategy is to pick values on a logarithmic scale. $0\le \sigma \le 1$ could be too narrow. There's no guarantee that any particular interval will do well in all problems. Trying $0.001, 0.1,1, 10, 100$ could be a useful starting place. – Sycorax Dec 14 '16 at 23:46
  • 1
    http://stats.stackexchange.com/questions/43943/which-search-range-for-determining-svm-optimal-parameters Also relevant – Sycorax Dec 16 '16 at 16:40

0 Answers0