# How to train/predict a very simple feed-forward neural network in Flux?

**URL:** <https://discourse.julialang.org/t/how-to-train-predict-a-very-simple-feed-forward-neural-network-in-flux/39534>\
**Category:** Machine Learning\
**Created:** [May 15, 2020, 4:57pm UTC](https://discourse.julialang.org/t/how-to-train-predict-a-very-simple-feed-forward-neural-network-in-flux/39534 "2020-05-15T16:57:40Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![sylvaticus](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sylvaticus/32/203883_2.png) [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)\
**Post date:** [May 15, 2020, 4:57pm UTC](https://discourse.julialang.org/t/how-to-train-predict-a-very-simple-feed-forward-neural-network-in-flux/39534/1 "2020-05-15T16:57:41Z")

</div>

I must be idiot but I can’t realise how to do very basic… no Convolution, RNN, gates,… just a basic plain feed-forward neural network design/testing/validation in Flux.

Somethink like:

```julia
l1 = FullyConnectedLayer(tanh,2,3,w=ones(3,2), wb=zeros(3))
l2 = FullyConnectedLayer(tanh,3,2, w=ones(2,3), wb=zeros(2))
l3 = FullyConnectedLayer(linearf,2,1, w=ones(1,2), wb=zeros(1))
mynn = buildNetwork([l1,l2,l3],squaredCost,name="Feed-forward Neural Network Model 1")

xtrain = [0.1 0.2; 0.3 0.5; 0.4 0.1; 0.5 0.4; 0.7 0.9; 0.2 0.1]
ytrain = [0.3; 0.8; 0.5; 0.9; 1.6; 0.3]
xtest = [0.5 0.6; 0.14 0.2; 0.3 0.7; 2.0 4.0]
ytest = [1.1; 0.36; 1.0; 6.0]

train!(mynn,xtrain,ytrain,maxepochs=10000,η=0.01,rshuffle=false,nMsgs=10)
errors(mynn,xtest,ytest) # 0.000196
for (i,r) in enumerate(eachrow(xtest))
  println("x: $r ŷ: $(predict(mynn,r)[1]) y: $(ytest[i])")
end

```

---

<div class="post-metadata">

**Author:** ![Iulian.Cioarca](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/iulian.cioarca/32/30166_2.png) [@Iulian.Cioarca](https://discourse.julialang.org/u/Iulian.Cioarca)\
**Post date:** [May 15, 2020, 5:08pm UTC](https://discourse.julialang.org/t/how-to-train-predict-a-very-simple-feed-forward-neural-network-in-flux/39534/2 "2020-05-15T17:08:21Z")

</div>

Did you check the Flux [model\_zoo](https://github.com/FluxML/model-zoo/blob/master/vision/mnist/mlp.jl) for examples? I see they updated it quite recently. They also have the [60min blitz](https://github.com/FluxML/model-zoo/blob/master/tutorials/60-minute-blitz.jl) which could be useful.

---

<div class="post-metadata">

**Author:** ![Albert\_Zevelev](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/albert_zevelev/32/11844_2.png) [@Albert\_Zevelev](https://discourse.julialang.org/u/Albert_Zevelev)\
**Post date:** [May 16, 2020, 3:19am UTC](https://discourse.julialang.org/t/how-to-train-predict-a-very-simple-feed-forward-neural-network-in-flux/39534/3 "2020-05-16T03:19:52Z")

</div>

@sylvaticus maybe my code will help:  
[Generic Function to train NN w/ Flux](https://discourse.julialang.org/t/generic-function-to-train-nn-w-flux/37208)

---

<div class="post-metadata">

**Author:** ![sylvaticus](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sylvaticus/32/203883_2.png) [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)\
**Post date:** [May 17, 2020, 11:50am UTC](https://discourse.julialang.org/t/how-to-train-predict-a-very-simple-feed-forward-neural-network-in-flux/39534/4 "2020-05-17T11:50:57Z")

</div>

Thank you, that’s the clearer example I could find. Still I can’t get it working with toy data:

```julia
using Flux
xtrain = [0.1 0.2; 0.3 0.5; 0.4 0.1; 0.5 0.4; 0.7 0.9; 0.2 0.1]
ytrain = [0.3; 0.8; 0.5; 0.9; 1.6; 0.3]
xtest = [0.5 0.6; 0.14 0.2; 0.3 0.7]
ytest = [1.1; 0.36; 1.0]
# Direct way. Error: "Output should be scalar":
model = Chain(Dense(2, 1))
loss(x, y) = Flux.mse,(model(x), y)
Flux.@epochs 200 Flux.train!(loss, model, Flux.Data.DataLoader(xtrain', ytrain'), ADAGrad())

# Using Albert_Zevelev's "f" function. Error: DimensionMismatch:
d = Flux.Data.DataLoader(xtrain', ytrain');
function f(d, XT, YT, XH, YH;
           m = Chain(Dense(size(XT,2), 1)), #Model/Activation
           ℓ = Flux.mse, #Loss: mse, crossentropy...
           # #Penalty: add later...
           opt = ADAGrad(), #Optimiser
           nE = 200 #Number epochs
          )
  loss(x, y) = ℓ(m(x), y)
  Flux.@epochs nE Flux.train!(loss, params(m), d, opt)
  IS = Flux.mse(m(XT'), YT') |> sqrt
  OS = Flux.mse(m(XH'), YH') |> sqrt
  return IS, OS
end
f(d, xtrain, ytrain, xtest, ytest, m = Chain(Dense(12,1)), nE= 200)

```

I am sorry I really believe there is a problem with the documentation (obviously it is my own personal opinion): I had a look several times to the model\_zoo, very useful… if you already know a bit, if you need a starting point for your own problem. But they are all implementation on specific areas, there isn’t there a “model zero” tutorial.  
I believe there should be a very very very trivial example like the one I am trying to solve. No need to load data from Boston housing data or MNIST dataset, at this time this is a distraction.  
No convolutional layers, recurrent neural networks, models with gates… just a plain model to show how to build a model, how to get predictions, how to train it and how to check performances.  
After several hours I can’t still do it in Flux, and I feel very frustrated :-/

---

<div class="post-metadata">

**Author:** ![lhnguyen-vn](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lhnguyen-vn/32/15449_2.png) [@lhnguyen-vn](https://discourse.julialang.org/u/lhnguyen-vn)\
**Post date:** [May 17, 2020, 1:07pm UTC](https://discourse.julialang.org/t/how-to-train-predict-a-very-simple-feed-forward-neural-network-in-flux/39534/5 "2020-05-17T13:07:20Z")

</div>

> [@sylvaticus](#):
>
> using Flux xtrain = [0.1 0.2; 0.3 0.5; 0.4 0.1; 0.5 0.4; 0.7 0.9; 0.2 0.1] ytrain = [0.3; 0.8; 0.5; 0.9; 1.6; 0.3] xtest = [0.5 0.6; 0.14 0.2; 0.3 0.7] ytest = [1.1; 0.36; 1.0] # Direct way. Error: “Output should be scalar”: model = Chain(Dense(2, 1)) loss(x, y) = Flux.mse,(model(x), y) Flux.@epochs 200 Flux.train!(loss, model, Flux.Data.DataLoader(xtrain’, ytrain’), ADAGrad())

I’m sorry that your experience has been less than ideal, but I found Flux’s documentation a really good starting point. The [training section](https://fluxml.ai/Flux.jl/stable/training/training/), for example, details how to set up a model.

Here’s how your toy example could be implemented:

```julia
using Flux

xtrain = [0.1 0.2; 0.3 0.5; 0.4 0.1; 0.5 0.4; 0.7 0.9; 0.2 0.1]
ytrain = [0.3; 0.8; 0.5; 0.9; 1.6; 0.3]
xtest = [0.5 0.6; 0.14 0.2; 0.3 0.7]
ytest = [1.1; 0.36; 1.0]

model = Dense(2, 1) # Use Chain if you want to stack layers

loss(x, y) = Flux.mse(model(x), y)
ps = params(model)
dataset = [(xtrain', ytrain')] # Use DataLoader for easy minibatching
opt = ADAGrad()

Flux.@epochs 100 Flux.train!(loss, ps, dataset, opt) 

```

The train loop also takes an additional keyword argument `cb` for callbacks. For instance, if you want to see how the loss improves each epoch:

```julia
cb = () -> println(loss(xtrain', ytrain'))
Flux.@epochs 100 Flux.train!(loss, ps, dataset, opt, cb = cb)

```

---

<div class="post-metadata">

**Author:** ![shawngiese](https://avatars.discourse-cdn.com/v4/letter/s/b9bd4f/32.png) [@shawngiese](https://discourse.julialang.org/u/shawngiese)\
**Post date:** [May 26, 2021, 8:33pm UTC](https://discourse.julialang.org/t/how-to-train-predict-a-very-simple-feed-forward-neural-network-in-flux/39534/6 "2021-05-26T20:33:58Z")

</div>

That is a great example and ran smoothly. I tried using DataLoader for batching but then I got errors about the dimensions.

```julia
data_batch = Flux.Data.DataLoader((xtrain', ytrain'), batchsize=6)
Flux.@epochs 100 Flux.train!(loss, ps, data_batch, opt) 
┌ Info: Epoch 1 
└ @ Main C:\Users\shawn\.julia\packages\Flux\6o4DQ\src\optimise\train.jl:135

DimensionMismatch("A has dimensions (13,13) but B has dimensions (2,6)")

```

---

<div class="post-metadata">

**Author:** ![shawngiese](https://avatars.discourse-cdn.com/v4/letter/s/b9bd4f/32.png) [@shawngiese](https://discourse.julialang.org/u/shawngiese)\
**Post date:** [May 26, 2021, 8:42pm UTC](https://discourse.julialang.org/t/how-to-train-predict-a-very-simple-feed-forward-neural-network-in-flux/39534/7 "2021-05-26T20:42:40Z")

</div>

DataLoader was supposed to be the following I think:

`data_batch = Flux.Data.DataLoader((xtrain’, ytrain), batchsize=6)`