Overview
- I have good experience with Flux in the past, but now have problems with fitting a 2-input, 1-output data set: the result is rather different from what I get with linear regression.
- Also, I struggle with doing a surface plot of the resulting Flux FNN model.
I’m using the following packages:
using Plots; pyplot();
clibrary(:colorcet)
using LaTeXStrings;
using LinearAlgebra;
using Flux;
using Statistics;
and have introduced some global variables for consistency of plotting:
M1 = :v
MS1 = 7
MC1 = :orange
LW1 = 2.5
LS1 = :solid
LA1 = 0.8
# Plotting colors
seed = colorant"blue"
COL = distinguishable_colors(6,seed)
Linear regression result
Data taken from REGRESSION - Linear Regression Datasets, see https://people.sc.fsu.edu/~jburkardt/datasets/regression/x09.txt
- Input x: (person weight [
kg], person age[years]) - Output y: person blood fat content `[?]´
X = [[84.,46],[73,20],[65,52],[70,30],[76,57],[69,25],[63,28],[72,36],
[79,57],[75,44],[27,24],[89,31],[65,52],[57,23],[59,60],[69,48],
[60,34],[79,51],[75,50],[82,34],[59,46],[67,23],[85,37],[55,40],
[63,30]] |> x->reduce(hcat,x)
#
Y = [354.,190,405,263,451,302,288,385,402,365,209,290,346,254,
395,434,220,374,308,220,311,181,274,303,244]'
#
x1 = range(minimum(X[1,:]),maximum(X[1,:]),length=50)
x2 = range(minimum(X[2,:]),maximum(X[2,:]),length=50);
#
#
plot(X[1,:],X[2,:],Y',seriestype=:scatter3d,markercolor=MC1,marker=M1,markersize=MS1,label="")
plot!(xlabel="Weight [kg]", ylabel="Age [years]", zlabel="Blood fat content")
plot!(title="Humans: blood fat vs. weight, age",camera=(-60,10))
Result:
Linear regression of plane:
ϕ(x) = [1,x[1],x[2]] # basis functions
Φ = [ϕ(X[:,i]) for i in 1:size(X,2)] |> x -> reduce(hcat,x) # regressor
β_lin = Y/Φ; # parameters
L_lin = norm(Y-β_lin*Φ); # loss
f_lin = (x1,x2) -> β_lin*ϕ([x1,x2]) # mapping for surface plot
#
plot(x1,x2,f_lin,seriestype=:surface,color=:blues)
plot!(X[1,:],X[2,:],Y',seriestype=:scatter,markersize=MS1,marker=M1,markercolor=MC1,label="data")
plot!(xlabel="Weight [kg]", ylabel="Age [years]", zlabel="Blood fat content")
plot!(title="Humans: blood fat vs. weight, age",camera=(-60,10))
leads to:
Flux and neural network fitting
data = [(X,Y)]
opt = ADAM(0.05, (0.99, 0.999));
#
#
nz = 5 # number of nodes in hidden layer
mod = Chain(Dense(2,nz,tanh),Dense(nz,1)) # 1 hidden layer, 2 inputs, 1 linear output
# Loss function
loss(x, y) = mean((mod(x).-y).^2)
# Extracting parameters from model
p = Flux.params(mod);
#
#
nE = 20_000 # number of epochs
for i in 1:nE
Flux.train!(loss,p,data,opt)
end
The results (using prompt julia> to differentiate inputs and outputs):
julia> Tracker.data(mod(X))
1×25 Array{Float32,2}:
310.72 310.72 310.72 310.72 310.72 … 310.72 310.72 310.72 310.72
Question: Why on earth does Flux give a totally flat/constant plane? Compare with the sloping plane from linear regression…
Flux and Plots
I really like the Plots way of doing surface plots where I can specify two input vectors and one scalar output mapping (as compared to, e.g., MATLAB), see the above code for plotting the plane from linear regression.
I’d like to plot the Flux mapping as a surface plot, doing something like:
plot(x1, x2, f_flux, seriestype=:surface)
but struggle with defining f_flux… The problem is related to defining f_flux with two scalar inputs. If I use a vector input, there is no problem to compute the output:
julia> f_flux_vector = x -> Tracker.data(mod(x))[:];
julia> f_flux_vector(X)
25-element Array{Float32,1}:
310.72003
310.72003
...
310.72003
But Plots needs a function with two scalar inputs. However, if I define f_flux with two scalar inputs, plot doesn’t work for me…
Question: how can I define the Flux FNN mapping with two inputs so that I can plot the surface?
Suggestions are appreciated!
