# Problems with Flux

**URL:** <https://discourse.julialang.org/t/problems-with-flux/9719>\
**Category:** Machine Learning\
**Created:** [March 14, 2018, 5:14pm UTC](https://discourse.julialang.org/t/problems-with-flux/9719 "2018-03-14T17:14:01Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![jacobcvt12](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jacobcvt12/32/2695_2.png) [@jacobcvt12](https://discourse.julialang.org/u/jacobcvt12)\
**Post date:** [March 14, 2018, 5:14pm UTC](https://discourse.julialang.org/t/problems-with-flux/9719/1 "2018-03-14T17:14:01Z")

</div>

I’m having trouble fitting a really basic neural network in Flux. I wanted to get a handle on the syntax by simulating simple data from a logistic regression, but the model isn’t converging to anything reasonable. I think this is probably due to a misunderstanding on my part of the syntax. Any thoughts on what I’m doing wrong? Code below

```julia
# generate logistic regression data
using Distributions

## size of data
n = 1000
d = 10 # covariates including bias

## seed
srand(1)

## coefficients
b = rand(Normal(), d)

## covariates
X = rand(Normal(), n, d)
X[:, 1] = ones(n)

## output
θ = sigmoid.(X * b) # true probabilities
y = rand.(Bernoulli.(θ))
mean((θ - y) .^ 2) # MSE with truth

## now drop bias from data
X = X[:, 2:d]

# set up neural network
using Flux 
using Flux.Tracker
using Flux: @epochs

model = Chain(Dense(d - 1, 5, sigmoid),
              Dense(5, 1, sigmoid))

# train model

## set up loss to minimize, optimizer, and data
loss(x, y) = Flux.mse(model(x), y)
opt = ADAM(params(model))
data = [(X', y)] # note the transpose of covariates!

loss(X', y) # 267.83

# now fit model
@epochs 100 Flux.train!(loss, data, opt)

# check model performance
loss(X', y) # 252.88

```

---

<div class="post-metadata">

**Author:** ![jacobcvt12](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jacobcvt12/32/2695_2.png) [@jacobcvt12](https://discourse.julialang.org/u/jacobcvt12)\
**Post date:** [March 14, 2018, 7:27pm UTC](https://discourse.julialang.org/t/problems-with-flux/9719/2 "2018-03-14T19:27:49Z")

</div>

I fixed the issue by minimizing crossentropy and using `onehotbatch` instead of of minimizing MSE.

```julia
# load packages
using Distributions
using Flux 
using Flux.Tracker
using Flux: onehotbatch, argmax, crossentropy, throttle, @epochs

# generate logistic regression data

## size of data
n = 1000
d = 10 # covariates including bias

## seed
srand(1)

## coefficients
b = rand(Normal(), d)

## covariates
X = rand(Normal(), n, d)
X[:, 1] = ones(n)

## output
θ = sigmoid.(X * b) # true probabilities
Y = onehotbatch(rand.(Bernoulli.(θ)), 0:1)

## now drop bias from data and transpose matrix
X = X[:, 2:d]
X = X'

# set up neural network

model = Chain(Dense(d - 1, 5, sigmoid),
              Dense(5, 2),
              softmax)

# train model

## set up loss to minimize, optimizer, and data
loss(x, y) = crossentropy(model(x), y)
opt = ADAM(params(model))
data = [(X, Y)] # note the transpose of covariates!

## accuracy
accuracy(x, y) = mean(argmax(model(x)) .== argmax(y))
accuracy(X, Y) # 32.5%

# now fit model
@epochs 1000 Flux.train!(loss, data, opt)

# check model performance
accuracy(X, Y) # 86.7%

```

---

<div class="post-metadata">

**Author:** ![jacobcvt12](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jacobcvt12/32/2695_2.png) [@jacobcvt12](https://discourse.julialang.org/u/jacobcvt12)\
**Post date:** [March 14, 2018, 8:10pm UTC](https://discourse.julialang.org/t/problems-with-flux/9719/3 "2018-03-14T20:10:54Z")

</div>

Circling back on this, I found that the issue was the dimension of the output… In my original post, X was of dimension `d x n` and y was dimension `n x 1`. As a side effect of changing y to one hot encoding, I corrected the dimension to `2 x n`.