Problems using meshgrid to plot a surface in an attempt to visualize feature mapping and SVM

New to Julia
, ,
1

Hi all,
I am working on a little project using toy data to visualize what happens when you map data to a higher feature space and perform SVM (Support Vector Machine) in this higher space. With the obtained set of weights corresponding to the decision boundary hyperplane, I’d like to map this hyperplane back into the original 2D space to obtain a non-linear decision boundary.

I stumbled onto two problems. To make myself clear I will discuss them along with my code.

Generating circles and 2D scatter plot

    data, distr_index = make_circles(100, noise=0.08, factor=0.1, random_state=1)
	dfcircles = DataFrame(data, :auto)
	scatter(dfcircles[:,1],dfcircles[:,2], color = distr_index, xlabel = "x1", ylabel 	="x2")

Since I’m a new user I can not put multiple pictures in a post.

3D Scatterplot

scatter(Z[:,1], Z[:,2], Z[:,3], color = distr_index)

Generating feature map

function feature_map(X)
	return df = DataFrame(x1 = (sqrt(2)*X[:,1].*X[:,2]), x2 = X[:,1].^2, x3 = X[:,2].^2)
	end
Z = feature_map(dfcircles)

SVM

Z_matrix = Z |> Matrix{Float64}
clf = SVC(C=1, kernel = "linear")
	fit!(clf, Z_matrix, distr_index)
w = collect(Iterators.flatten(clf.coef_))
b = collect(Iterators.flatten(clf.intercept_))

create x,y

xx, yy = meshgrid(-1:0.02:1, 0:0.02:1)

calculate corresponding z

boundary = ((xx .*(-w[1]) - (yy .* w[2])) .- b[1]) .* (1 / w[2])

3 D scatter plot plus surface

scatter(Z[:,1], Z[:,2], Z[:,3], color = distr_index)
surface!(xx, yy, boundary)

Result:

azertyuio
azertyuio755×483 123 KB

Problem 1: I don’t want it to be a heatmap (rather just a full hyperplane), what should I use instead of meshgrid? My final result should look something like this:
(Link.

Now onto the plotting of the decision boundary:
Initialize data

h = 0.01 #stepsize meshgrid
	x_min, x_max = minimum(Z_matrix[:, 1]) - .5, maximum(Z_matrix[:, 1]) + .5
    y_min, y_max = minimum(Z_matrix[:, 2]) - .5, maximum(Z_matrix[:, 2]) + .5
    xx2, yy2 = meshgrid(x_min:h:x_max, y_min:h:y_max)

Predicting on meshgrid:
Now this is where it also goes wrong. I’m following python code from a blog post (Link.
In here the following python code is performed:

Z = clf.predict(np.c_[xx.ravel(), yy.ravel()])

Problem 2: Do you know what’s the julia equivalent for this?
I tried this (It comes from an other blogpost but it doesn’t recognize “predict_proba” and gives an error for decision_function)

Z2 = try 
            # Not implemented for some
            decision_function(clf, hcat(xx[:], yy[:]))
        catch
            predict_proba(clf, hcat(xx[:], yy[:]))[:, 2]
        end

Thank you in advance to anyone that can help. I’m new to Julia and to relatively new to coding in general.
These are the packages (not all used) that are in my notebook.

using ScikitLearn, DataFrames, Plots, KernelFunctions, LinearAlgebra, ScikitLearn.CrossValidation: train_test_split, ScikitLearn.Utils: meshgrid
@sk_import datasets: make_circles
	@sk_import svm: SVC
	@sk_import preprocessing: StandardScaler