Julia Flux, how to build a simple multilayer perception to solve xor problem

General Usage
1

Hello. Someone knows how to build a simple multilayer perceptron with flux to solve a xor problem? The examples are very complex to start with flux. May someone help?

1 Like
2

Maybe this will hell

https://www.github.com/FluxML/model-zoo/tree/master/other%2Fhousing%2Fhousing.jl

1 Like
3

I was thinking in something simple. Just for a xor problem. :slightly_smiling_face:

4

It’s your lucky day :smile: It’s one of the first examples I made while exploring Flux:

using Flux

## Taking gradients
# f(x) = 3x^2 + 2x + 1
# df(x) = Flux.gradient(f, x)[1]
# f(5)
# df(5)

## Input values for the XOR
# X = Matrix{Float32}([0 0; 0 1; 1 0; 1 1])
X = [0f0 0 1 1; 0 1 0 1]
y = [0f0 1 1 0]

## Define the XOR gated model
xornn_model = Chain(
    Dense(2, 2, sigmoid),
    Dense(2, 1, sigmoid)
)

## Check the parameters of the xornn_model
params(xornn_model)
# Matrix of parameters of the first layer
params(xornn_model)[1]
# Matrix of parameters of the second final neuron
params(xornn_model)[3]

## Define the loss function as the MSE 
loss_fn(x, y) = Flux.mse(xornn_model(x), y)

# Create an optimizer for the gradient descent algorithm
# opt = Descent(0.01)
opt = ADAM(0.1)

## Train the model
# To train the network for 1 epoch we can use train! function.
# Here, we are training over N epochs with Batch Gradient Descent
N = 500
loss = zeros(500)
for i in 1:500
    Flux.train!(loss_fn, params(xornn_model), [(X, y)], opt)
    loss[i] = loss_fn(X, y)
    if i % 10 == 0
        println(loss[i])
    end
end 


## Check to see if model is predicting nice
# Loss should be VERY close to zero, if not, you may have to try
# again initializing the network with some other random parameters
# e.g redefine the xornn_model
loss_fn(X, y)

# Check the output
xornn_model(X)

# Output should be something like this
# julia> xornn_model(X)
# 1×4 Array{Float32,2}:
#  0.018062  0.984418  0.984774  0.0149787
# ... in another run I got:
# 1×4 Array{Float32,2}:
#  0.0177666  0.984167  0.979524  0.0151272


## Some minimal inspection graph
using Plots
plot(1:N, loss)
title!("MSE vs epochs of XOR network")
6 Likes
5

Thandks @r2cp and @danielw2904 for the response. They helped a lot. :slightly_smiling_face:

6

beautiful minimal example!

I have prepared a Jupyter notebook using this example here.

2 Likes
7

Also, did one repository here :slight_smile:
https://github.com/jcbritobr/flux-xor-ml

1 Like