Hello. Someone knows how to build a simple multilayer perceptron with flux to solve a xor problem? The examples are very complex to start with flux. May someone help?
1 Like
Maybe this will hell
https://www.github.com/FluxML/model-zoo/tree/master/other%2Fhousing%2Fhousing.jl
1 Like
I was thinking in something simple. Just for a xor problem. 
It’s your lucky day 
It’s one of the first examples I made while exploring Flux:
using Flux
## Taking gradients
# f(x) = 3x^2 + 2x + 1
# df(x) = Flux.gradient(f, x)[1]
# f(5)
# df(5)
## Input values for the XOR
# X = Matrix{Float32}([0 0; 0 1; 1 0; 1 1])
X = [0f0 0 1 1; 0 1 0 1]
y = [0f0 1 1 0]
## Define the XOR gated model
xornn_model = Chain(
Dense(2, 2, sigmoid),
Dense(2, 1, sigmoid)
)
## Check the parameters of the xornn_model
params(xornn_model)
# Matrix of parameters of the first layer
params(xornn_model)[1]
# Matrix of parameters of the second final neuron
params(xornn_model)[3]
## Define the loss function as the MSE
loss_fn(x, y) = Flux.mse(xornn_model(x), y)
# Create an optimizer for the gradient descent algorithm
# opt = Descent(0.01)
opt = ADAM(0.1)
## Train the model
# To train the network for 1 epoch we can use train! function.
# Here, we are training over N epochs with Batch Gradient Descent
N = 500
loss = zeros(500)
for i in 1:500
Flux.train!(loss_fn, params(xornn_model), [(X, y)], opt)
loss[i] = loss_fn(X, y)
if i % 10 == 0
println(loss[i])
end
end
## Check to see if model is predicting nice
# Loss should be VERY close to zero, if not, you may have to try
# again initializing the network with some other random parameters
# e.g redefine the xornn_model
loss_fn(X, y)
# Check the output
xornn_model(X)
# Output should be something like this
# julia> xornn_model(X)
# 1×4 Array{Float32,2}:
# 0.018062 0.984418 0.984774 0.0149787
# ... in another run I got:
# 1×4 Array{Float32,2}:
# 0.0177666 0.984167 0.979524 0.0151272
## Some minimal inspection graph
using Plots
plot(1:N, loss)
title!("MSE vs epochs of XOR network")
6 Likes
