How to manually calculate predictions of kernlabs SVM

Question

I am trying to manually replicate the predictions of kernlabs SVM (polynomial & radial kernel) using caret. Here is the code to fit the model:

  fitControlSVM <- trainControl(
    method = "repeatedcv",
    number = 5,
    repeats = 5,
    summaryFunction = twoClassSummary,
    classProbs = TRUE,
    allowParallel = TRUE)
  
  data("PimaIndiansDiabetes2", package = "mlbench")
  pima.data <- na.omit(PimaIndiansDiabetes2)
  
  set.seed(1)
  svm_poly <- train(diabetes ~ .,
                      data = pima.data,
                      method = "svmPoly",
                      trControl = fitControlSVM,
                      metric = "Spec",
                      preProcess = c("center","scale"),
                      tuneLength = 3) # tunelength creates a grid

Given the fit, I can extract the coefficients & xmat of the "best" model:

coefs <- svm_poly$finalModel@coef[[1]]
xmat <- svm_poly$finalModel@xmatrix[[1]]

and quickly get the coefficient for each predictor:

(predictor <- coefs %*% xmat)

given the svm_poly$finalModel, I further get the hyperparameter:

SV type: C-svc  (classification) 
 parameter : cost C = 0.25 

Polynomial kernel function. 
 Hyperparameters : degree =  1  scale =  0.01  offset =  1

But I just don't know/understand how to calculate the prediction myself, as I am not that confident with SVM to be honest... So I am trying to replicate manually:

predict(svm_poly, pima.data[,-9], type ="prob")

Thanks

GMSL · Answer 1 · 2021-05-24T13:20:19.940

This Cross Validated answer suggests that (quote):

"For data point x your SVM calculates decision value d in the following way:"

d <- sum(w * x) + b

"If d>0 then label of x is +1, else it's −1."

So I believe that the answer is thus. Given you have a trained C-SVM model object model from kernlab (your svmPoly caret method trains kernlab SVMs), and you want to predict the class of a single new datapoint x_new:

x_mat <- model@xmatrix[[1]]
coeffs <- model@coef[[1]]
b <- model@b
x <- x_mat %*% t(x_new)
d <- (coeffs %*% x) + b

If d > 0 then the prediction is one class, otherwise it is the other class.

I'm pretty sure this is correct but not 100 % sure so I would appreciate any comments from those who are more knowledgable.

How to manually calculate predictions of kernlabs SVM

1 Answers1