Multilayer perceptron with multidimensional output array using Flux

Specific Domains Machine Learning
1

Hi, I have some problems coding a multilayer perceptron using Flux. My data is given in the form

D=(x_i, y_i)_{i=1}^n where x_i is a scalar and y_i is a 3-dim vector . I want to train a neural network using flux. Due to my data, my input is a vector of scalars and my output is a vector of vectors i.e. x = (x_i) and y=(y_i), where x has dimension n and y has dimension n \times 3.

Sadly I have problems defining a model with an output of the form n\times3. I get the error:

ERROR: LoadError: MethodError: no method matching (::Dense{typeof(σ), Matrix{Float32}, Vector{Float32}})(::Float64)

My code is the following. (I have the feeling that I have to iterate thourgh the model somehow?)

using Flux: mse
using MLJ: partition

# Generating #n data points of the form (Xi, Yi) with Xi a scalar Yi a vector of size 3
n = 10
X = rand(n)  # input of size n
Y = rand(n, 3) # output of size nx3

# Create data batch for train and test
train, test = partition(eachindex(X), 0.7, rng=333)
xtrn,  ytrn = X[train], Y[train, :]  
xtst,  ytst = X[test], Y[test, :]

function flux(xtrn, ytrn, xtst, ytst; opt, nepochs)
    data = [(xtrn[i], ytrn[i,:]) for i in 1:length(xtrn)]
    Nx, Ny = size(xtrn,1),  size(ytrn,1)  # number of input and output i.e. 7
    Nnrns = trunc(Int, 2/3*(Nx+Ny))  #number of neurons in hidden layer

    model = Chain(Dense(Nx, Nnrns, sigmoid), Dense(Nnrns, Ny, sigmoid), Dense(Ny, Ny, identity));

    # Loss/cost function
    loss(x, y) = Flux.mse(model(x), y)

    # Parameters of the model
    para = Flux.params(model);
    trnloss = [];
    tstloss = [];

    for epoch = 1:nepochs
        Flux.train!(loss,para,data,opt)
        append!(trnloss, loss(xtrn, ytrn))
        append!(tstloss, loss(xtst, ytst))
    end
    return trnloss, tstloss
end

println(" ", "______________________________________")
Opt_All =[ADAM(), ADAMW(), ADAGrad(0.1), AdaMax(), ADADelta(0.9), AMSGrad(),
           NADAM(), RADAM(), Descent(0.1), Flux.Nesterov(), RMSProp(),
           Flux.Momentum()];
Opt_String =["ADAM", "ADAMW", "ADAGRAD", "AdaMax", "ADADelta", "AMSGrad", "NADAM", "RADAM", "Descent", "Neterov", "RMSProp", "Momentum"]

for (opt, optname) in zip(Opt_All, Opt_String)
    nepochs=30
    trnloss, tstloss = flux(xtrn, ytrn, xtst, ytst; opt, nepochs)
end
2

Note that Flux.jl expects data in the format nfeatures x nsamples. This is different from most machine learning frameworks and different from what you did.

using Flux
nsamples = 10; in_features = 6; out_features = 3;
X = randn(Float32, in_features, nsamples);
Y = randn(Float32, out_features, nsamples);
data = Flux.Data.DataLoader((X,Y), batchsize=4); # automatic batching convenience
X1, Y1 = first(data); 
@show size(X1) size(Y1)

If you really want to feed your model with some other kind of data format or non array types then you can use the pattern:

model = Flux.Chain(to_flux_format, some, flux, layers, from_flux_format)
3

Thanks for the advice. I edited the code:

using Flux
nsamples = 10; in_features = 1; out_features = 3;
X = randn(Float32, in_features, nsamples);
Y = randn(Float32, out_features, nsamples);
data = Flux.Data.DataLoader((X,Y), batchsize=4); # automatic batching convenience

X1, Y1 = first(data);
@show size(X1) size(Y1)

Nx = size(X1, 1)
Ny = size(Y1, 1)
Nnrns = trunc(Int, 2/3*(Nx+Ny))
model = Chain(Dense(Nx, Nnrns, sigmoid), Dense(Nnrns, Ny, sigmoid), Dense(Ny, Ny, identity));

# Loss/cost function
loss(x, y) = Flux.mse(model(x), y)

# Parameters of the model
para = Flux.params(model);

trnloss = [];
opt = ADAM()

for epoch = 1:50
    Flux.train!(loss,para,data,opt)
    append!(trnloss, loss(X1, Y1))
end