Hi all,

I am working on a little project using toy data to visualize what happens when you map data to a higher feature space and perform SVM (Support Vector Machine) in this higher space. With the obtained set of weights corresponding to the decision boundary hyperplane, I’d like to map this hyperplane back into the original 2D space to obtain a non-linear decision boundary.

I stumbled onto two problems. To make myself clear I will discuss them along with my code.

Generating circles and 2D scatter plot

```
data, distr_index = make_circles(100, noise=0.08, factor=0.1, random_state=1)
dfcircles = DataFrame(data, :auto)
scatter(dfcircles[:,1],dfcircles[:,2], color = distr_index, xlabel = "x1", ylabel ="x2")
```

Since I’m a new user I can not put multiple pictures in a post.

3D Scatterplot

```
scatter(Z[:,1], Z[:,2], Z[:,3], color = distr_index)
```

Generating feature map

```
function feature_map(X)
return df = DataFrame(x1 = (sqrt(2)*X[:,1].*X[:,2]), x2 = X[:,1].^2, x3 = X[:,2].^2)
end
Z = feature_map(dfcircles)
```

SVM

```
Z_matrix = Z |> Matrix{Float64}
clf = SVC(C=1, kernel = "linear")
fit!(clf, Z_matrix, distr_index)
w = collect(Iterators.flatten(clf.coef_))
b = collect(Iterators.flatten(clf.intercept_))
```

create x,y

```
xx, yy = meshgrid(-1:0.02:1, 0:0.02:1)
```

calculate corresponding z

```
boundary = ((xx .*(-w[1]) - (yy .* w[2])) .- b[1]) .* (1 / w[2])
```

3 D scatter plot plus surface

```
scatter(Z[:,1], Z[:,2], Z[:,3], color = distr_index)
surface!(xx, yy, boundary)
```

Result:

**Problem 1**: I don’t want it to be a heatmap (rather just a full hyperplane), what should I use instead of meshgrid? My final result should look something like this:

(Link.

Now onto the plotting of the decision boundary:

Initialize data

```
h = 0.01 #stepsize meshgrid
x_min, x_max = minimum(Z_matrix[:, 1]) - .5, maximum(Z_matrix[:, 1]) + .5
y_min, y_max = minimum(Z_matrix[:, 2]) - .5, maximum(Z_matrix[:, 2]) + .5
xx2, yy2 = meshgrid(x_min:h:x_max, y_min:h:y_max)
```

Predicting on meshgrid:

Now this is where it also goes wrong. I’m following python code from a blog post (Link.

In here the following python code is performed:

```
Z = clf.predict(np.c_[xx.ravel(), yy.ravel()])
```

**Problem 2**: Do you know what’s the julia equivalent for this?

I tried this (It comes from an other blogpost but it doesn’t recognize “predict_proba” and gives an error for decision_function)

```
Z2 = try
# Not implemented for some
decision_function(clf, hcat(xx[:], yy[:]))
catch
predict_proba(clf, hcat(xx[:], yy[:]))[:, 2]
end
```

Thank you in advance to anyone that can help. I’m new to Julia and to relatively new to coding in general.

These are the packages (not all used) that are in my notebook.

```
using ScikitLearn, DataFrames, Plots, KernelFunctions, LinearAlgebra, ScikitLearn.CrossValidation: train_test_split, ScikitLearn.Utils: meshgrid
@sk_import datasets: make_circles
@sk_import svm: SVC
@sk_import preprocessing: StandardScaler
```