3

We're working with a similar dual SVR problem that involves the inversion of a Gram (kernel) matrix:

$\boldsymbol{S}_{i,j} = e^{ -\gamma ||\vec{x_i} - \vec{x_j}||_2^2}$

With some data-sets (e.g.: UCI ForestFires) the inversion of $\boldsymbol{S}$ is not always possible, also after the removal double examples. So, why this matrix is singular?

Thanks.

Filippo Portera
  • 101
  • 1
  • 6
  • The simplest work-around is to use $S +\lambda I$ for some small $\lambda > 0 $. This assures that the sum is PD and therefore invertable. – Sycorax Jun 11 '21 at 13:40
  • 3
    This covariance model is particularly problematic when one or more of the points are sufficiently separated from the others, because the corresponding entries of $S$ are essentially zero. You get either a singular matrix or a very poorly conditioned (i.e., near-singular) one. Adding a multiple of the identity can dramatically change certain calculations based on $S,$ so take that advice with care. – whuber Jun 11 '21 at 14:53
  • 3
    There are a couple of votes to move this to the maths SE but I think it's a good fit here, particularly since the OP may be interested in any statistical implications of @whuber's point – Silverfish Jun 12 '21 at 13:21

0 Answers0