How do I specify a precomputed kernel in SVM using LIBSVM or MLJ?

Specific Domains Machine Learning
1

Question arising in an MLJ slack post:

maybe this is a naive question, but how do you actually train an SVM using a precomputed kernel using LIBSVM or MLJ? I can’t seem to find a way to “give” the kernel matrix to any sort of fit function or model as a parameter

1 Like
2

This is not clear, even in the LIBSVM (C-code) docs, as far as I can tell. However, I found this document which explains the kernel should be provided in a file, which is specified as a command-line argument in the C-case.

Looking at the LIBSVM.jl code, I can’t find an interface point, which means there won’t be one in the MLJ wrapper either, as this is based on the former.

I’ve opened an issue at LIBSVM.jl querying this.

Happy to add the interface point to the MLJ wrapper when this is clarified.

@mpastell

2 Likes
3

It’s long time, find this way

snippets based on MLJ SVM tutorial

import MLJ: fit!,predict

using MLJ
using Plots
using PrettyPrinting
using Random
using KernelFunctions

#define kernelmethods
k1=PolynomialKernel(; degree=2, c=1)
k2 = SqExponentialKernel() ∘ ScaleTransform(1.5)

# make data
n1=n2=10
Random.seed!(3203)
X = randn(20, 2)
y=vcat(fill(-1, n1), fill(1, n2))
xs,ys=X[:,1],X[:,2]
#scatter(X[:,1],X[:,2],group=y,label=false)

X = MLJ.table(X)
y = categorical(y);

# work flow 
@time SVC = @load SVC pkg=LIBSVM

svc_mdl = SVC(kernel=k2) #<=== kernel is here

svc = machine(svc_mdl, X, y)

fit!(svc);

ypred =predict(svc, X)

misclassification_rate(ypred, y)
4

and Use this way to find more detail docs

using MLJ     # or MLJModels 
doc("NuSVC", pkg="LIBSVM")

User-defined kernels
  ======================

  k(x1, x2) = x1'*x2 # equivalent to `LIBSVM.Kernel.Linear`
  model = NuSVC(kernel=k)
  mach = machine(model, X, y) |> fit!
  
  julia> yhat = predict(mach, Xnew)
5

As for kernel matrix

please look at this

support-vector-machine

using KernelFunctions
using LIBSVM

# custom kernel function
k = SqExponentialKernel() ∘ ScaleTransform(1.5)

# configure  model
model = svmtrain(kernelmatrix(k, x_train), y_train; kernel=LIBSVM.Kernel.Precomputed)

# predict with model

y_pred, _ = svmpredict(model, kernelmatrix(k, x_train, x_test));