### Overview

- I have good experience with Flux in the past, but now have problems with fitting a 2-input, 1-output data set: the result is rather different from what I get with linear regression.
- Also, I struggle with doing a surface plot of the resulting Flux FNN model.

I’m using the following packages:

```
using Plots; pyplot();
clibrary(:colorcet)
using LaTeXStrings;
using LinearAlgebra;
using Flux;
using Statistics;
```

and have introduced some global variables for consistency of plotting:

```
M1 = :v
MS1 = 7
MC1 = :orange
LW1 = 2.5
LS1 = :solid
LA1 = 0.8
# Plotting colors
seed = colorant"blue"
COL = distinguishable_colors(6,seed)
```

### Linear regression result

Data taken from https://people.sc.fsu.edu/~jburkardt/datasets/regression/regression.html, see https://people.sc.fsu.edu/~jburkardt/datasets/regression/x09.txt

- Input x: (person weight [
`kg`

], person age`[years]`

) - Output y: person blood fat content `[?]´

```
X = [[84.,46],[73,20],[65,52],[70,30],[76,57],[69,25],[63,28],[72,36],
[79,57],[75,44],[27,24],[89,31],[65,52],[57,23],[59,60],[69,48],
[60,34],[79,51],[75,50],[82,34],[59,46],[67,23],[85,37],[55,40],
[63,30]] |> x->reduce(hcat,x)
#
Y = [354.,190,405,263,451,302,288,385,402,365,209,290,346,254,
395,434,220,374,308,220,311,181,274,303,244]'
#
x1 = range(minimum(X[1,:]),maximum(X[1,:]),length=50)
x2 = range(minimum(X[2,:]),maximum(X[2,:]),length=50);
#
#
plot(X[1,:],X[2,:],Y',seriestype=:scatter3d,markercolor=MC1,marker=M1,markersize=MS1,label="")
plot!(xlabel="Weight [kg]", ylabel="Age [years]", zlabel="Blood fat content")
plot!(title="Humans: blood fat vs. weight, age",camera=(-60,10))
```

Result:

Linear regression of plane:

```
ϕ(x) = [1,x[1],x[2]] # basis functions
Φ = [ϕ(X[:,i]) for i in 1:size(X,2)] |> x -> reduce(hcat,x) # regressor
β_lin = Y/Φ; # parameters
L_lin = norm(Y-β_lin*Φ); # loss
f_lin = (x1,x2) -> β_lin*ϕ([x1,x2]) # mapping for surface plot
#
plot(x1,x2,f_lin,seriestype=:surface,color=:blues)
plot!(X[1,:],X[2,:],Y',seriestype=:scatter,markersize=MS1,marker=M1,markercolor=MC1,label="data")
plot!(xlabel="Weight [kg]", ylabel="Age [years]", zlabel="Blood fat content")
plot!(title="Humans: blood fat vs. weight, age",camera=(-60,10))
```

leads to:

### Flux and neural network fitting

```
data = [(X,Y)]
opt = ADAM(0.05, (0.99, 0.999));
#
#
nz = 5 # number of nodes in hidden layer
mod = Chain(Dense(2,nz,tanh),Dense(nz,1)) # 1 hidden layer, 2 inputs, 1 linear output
# Loss function
loss(x, y) = mean((mod(x).-y).^2)
# Extracting parameters from model
p = Flux.params(mod);
#
#
nE = 20_000 # number of epochs
for i in 1:nE
Flux.train!(loss,p,data,opt)
end
```

The results (using prompt `julia> `

to differentiate inputs and outputs):

```
julia> Tracker.data(mod(X))
1×25 Array{Float32,2}:
310.72 310.72 310.72 310.72 310.72 … 310.72 310.72 310.72 310.72
```

**Question**: Why on earth does Flux give a totally flat/constant plane? Compare with the sloping plane from linear regression…

### Flux and Plots

I really like the Plots way of doing surface plots where I can specify two input vectors and one scalar output mapping (as compared to, e.g., MATLAB), see the above code for plotting the plane from linear regression.

I’d like to plot the Flux mapping as a surface plot, doing something like:

```
plot(x1, x2, f_flux, seriestype=:surface)
```

but struggle with defining `f_flux`

… The problem is related to defining `f_flux`

with two scalar inputs. If I use a vector input, there is no problem to compute the output:

```
julia> f_flux_vector = x -> Tracker.data(mod(x))[:];
julia> f_flux_vector(X)
25-element Array{Float32,1}:
310.72003
310.72003
...
310.72003
```

But Plots needs a function with two scalar inputs. However, if I define `f_flux`

with two scalar inputs, `plot`

doesn’t work for me…

**Question**: how can I define the Flux FNN mapping with two inputs so that I can plot the surface?

**Suggestions are appreciated!**