# Julia Flux, how to build a simple multilayer perception to solve xor problem

**URL:** <https://discourse.julialang.org/t/julia-flux-how-to-build-a-simple-multilayer-perception-to-solve-xor-problem/48901>\
**Category:** General Usage\
**Tags:** flux\
**Created:** [October 23, 2020, 6:15pm UTC](https://discourse.julialang.org/t/julia-flux-how-to-build-a-simple-multilayer-perception-to-solve-xor-problem/48901 "2020-10-23T18:15:48Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![jcbritobr](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jcbritobr/32/219275_2.png) [@jcbritobr](https://discourse.julialang.org/u/jcbritobr)\
**Post date:** [October 23, 2020, 6:15pm UTC](https://discourse.julialang.org/t/julia-flux-how-to-build-a-simple-multilayer-perception-to-solve-xor-problem/48901/1 "2020-10-23T18:15:48Z")

</div>

Hello. Someone knows how to build a simple multilayer perceptron with flux to solve a xor problem? The examples are very complex to start with flux. May someone help?

---

<div class="post-metadata">

**Author:** ![danielw2904](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/danielw2904/32/10890_2.png) [@danielw2904](https://discourse.julialang.org/u/danielw2904)\
**Post date:** [October 23, 2020, 9:43pm UTC](https://discourse.julialang.org/t/julia-flux-how-to-build-a-simple-multilayer-perception-to-solve-xor-problem/48901/2 "2020-10-23T21:43:36Z")

</div>

Maybe this will hell

[https://www.github.com/FluxML/model-zoo/tree/master/other%2Fhousing%2Fhousing.jl](https://www.github.com/FluxML/model-zoo/tree/master/other%2Fhousing%2Fhousing.jl)

---

<div class="post-metadata">

**Author:** ![jcbritobr](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jcbritobr/32/219275_2.png) [@jcbritobr](https://discourse.julialang.org/u/jcbritobr)\
**Post date:** [October 23, 2020, 10:28pm UTC](https://discourse.julialang.org/t/julia-flux-how-to-build-a-simple-multilayer-perception-to-solve-xor-problem/48901/3 "2020-10-23T22:28:48Z")

</div>

I was thinking in something simple. Just for a xor problem. 🙂

---

<div class="post-metadata">

**Author:** ![r2cp](https://avatars.discourse-cdn.com/v4/letter/r/67e7ee/32.png) [@r2cp](https://discourse.julialang.org/u/r2cp)\
**Post date:** [October 23, 2020, 10:31pm UTC](https://discourse.julialang.org/t/julia-flux-how-to-build-a-simple-multilayer-perception-to-solve-xor-problem/48901/4 "2020-10-23T22:31:47Z")

</div>

It’s your lucky day 😄 It’s one of the first examples I made while exploring Flux:

```julia
using Flux

## Taking gradients
# f(x) = 3x^2 + 2x + 1
# df(x) = Flux.gradient(f, x)[1]
# f(5)
# df(5)

## Input values for the XOR
# X = Matrix{Float32}([0 0; 0 1; 1 0; 1 1])
X = [0f0 0 1 1; 0 1 0 1]
y = [0f0 1 1 0]

## Define the XOR gated model
xornn_model = Chain(
    Dense(2, 2, sigmoid),
    Dense(2, 1, sigmoid)
)

## Check the parameters of the xornn_model
params(xornn_model)
# Matrix of parameters of the first layer
params(xornn_model)[1]
# Matrix of parameters of the second final neuron
params(xornn_model)[3]

## Define the loss function as the MSE 
loss_fn(x, y) = Flux.mse(xornn_model(x), y)

# Create an optimizer for the gradient descent algorithm
# opt = Descent(0.01)
opt = ADAM(0.1)

## Train the model
# To train the network for 1 epoch we can use train! function.
# Here, we are training over N epochs with Batch Gradient Descent
N = 500
loss = zeros(500)
for i in 1:500
    Flux.train!(loss_fn, params(xornn_model), [(X, y)], opt)
    loss[i] = loss_fn(X, y)
    if i % 10 == 0
        println(loss[i])
    end
end 

## Check to see if model is predicting nice
# Loss should be VERY close to zero, if not, you may have to try
# again initializing the network with some other random parameters
# e.g redefine the xornn_model
loss_fn(X, y)

# Check the output
xornn_model(X)

# Output should be something like this
# julia> xornn_model(X)
# 1×4 Array{Float32,2}:
# 0.018062 0.984418 0.984774 0.0149787
# ... in another run I got:
# 1×4 Array{Float32,2}:
# 0.0177666 0.984167 0.979524 0.0151272

## Some minimal inspection graph
using Plots
plot(1:N, loss)
title!("MSE vs epochs of XOR network")

```

---

<div class="post-metadata">

**Author:** ![jcbritobr](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jcbritobr/32/219275_2.png) [@jcbritobr](https://discourse.julialang.org/u/jcbritobr)\
**Post date:** [October 23, 2020, 10:41pm UTC](https://discourse.julialang.org/t/julia-flux-how-to-build-a-simple-multilayer-perception-to-solve-xor-problem/48901/5 "2020-10-23T22:41:15Z")

</div>

Thandks @r2cp and @danielw2904 for the response. They helped a lot. 🙂

---

<div class="post-metadata">

**Author:** ![jbytecode](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jbytecode/32/17719_2.png) [@jbytecode](https://discourse.julialang.org/u/jbytecode)\
**Post date:** [January 28, 2022, 5:53pm UTC](https://discourse.julialang.org/t/julia-flux-how-to-build-a-simple-multilayer-perception-to-solve-xor-problem/48901/6 "2022-01-28T17:53:21Z")

</div>

beautiful minimal example!

I have prepared a Jupyter notebook using this example [here](https://github.com/jbytecode/notebooks/blob/main/flux-xor-nn.ipynb).

---

<div class="post-metadata">

**Author:** ![jcbritobr](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jcbritobr/32/219275_2.png) [@jcbritobr](https://discourse.julialang.org/u/jcbritobr)\
**Post date:** [March 21, 2022, 1:42pm UTC](https://discourse.julialang.org/t/julia-flux-how-to-build-a-simple-multilayer-perception-to-solve-xor-problem/48901/7 "2022-03-21T13:42:03Z")

</div>

Also, did one repository here 🙂  
[https://github.com/jcbritobr/flux-xor-ml](https://github.com/jcbritobr/flux-xor-ml)