I’m currently training an ANN with 54 inputs and 12 outputs and I have already achieved good results by using the following model:

model = Chain(Dense(54,54,sigmoid), Dense(54,54,sigmoid), Dense(54,12,leakyrelu)).

However, I’m trying to apply regularization, in order to improve my results. I’m currently using the mse loss function. I tried to implement regularization by doing:

opt = Optimiser(WeightDecay(lambda), ADAGrad()),

and I set lambda=1, but I’m not getting better results. Any ideias about how I could implement regularization?

Here’s my code for the ANN training:

```
function flux_training(x_train::Array{Float64,2}, y_train::Array{Float64,2}, n_epochs::Int, lambda::Int)
model = Chain(Dense(54,54,sigmoid),Dense(54,54,sigmoid),Dense(54,12,leakyrelu))
loss(x,y) = Flux.mse(model(x),y)
ps = params(model)
dataset = Flux.Data.DataLoader(x_train', y_train', batchsize = 32, shuffle = true)
opt = Optimiser(WeightDecay(lambda), ADAGrad())
evalcb() = @show(loss(x_train', y_train'))
for epoch in 1:n_epochs
println("Epoch $epoch")
time = @elapsed Flux.train!(loss, ps, dataset, opt, cb=throttle(evalcb,3))
end
y_hat = model(x_train')'
return y_hat, model
end
```