Logistic regression in flux

New to Julia
1

I’m trying to implement binary logistic regression in flux. I’m simplifying the mlp.jl example from the model zoo.. I have tried 4 versions, only 1 works, please explain why.

V1.

D = 2; N = 20;
X = randn(D, N)
y = rand([0,1], N)
D, N = size(X)
data = repeated((X,Y),1)

### changeable part
model = Dense(D, 1, sigmoid)
loss(x, y) = Flux.binarycrossentropy(model(x), y)
Y = y
###

callback() = @show(loss(X, Y))
opt = Flux.Optimise.ADAM()
Flux.train!(loss, params(model), data, opt, cb = callback)

Fails:

LoadError: no method matching eps(::TrackedArray{…,Array{Float32,2}})

V2: same as above but

model = Chain(Dense(D, 1), sigmoid)

Fails:

DimensionMismatch("multiplicative identity defined only for square matrices")

V3: same as above but

model = Dense(D, 2, softmax) 
loss(x, y) = crossentropy(model(x), y)
Y = onehotbatch(y, 0:1)

Fails:

  MethodError: no method matching similar(::Float32)

V4: success!

model = Chain(Dense(D, 2), softmax)
loss(x, y) = crossentropy(model(x), y)
Y = onehotbatch(y, 0:1)

I understand why V4 works, but why not V1-V3?

3 Likes
2

@murphyk the following works on my computer:

using Flux
D = 2; N = 20;
X = randn(D, N)
y = rand([0,1], N)
D, N = size(X)
data = Iterators.repeated((X,y),1)

V1: Change Flux.binarycrossentropy to Flux.crossentropy

model = Dense(D, 1, sigmoid)
loss(x, y) = Flux.crossentropy(model(x), y)
Y = y

callback() = @show(loss(X, Y))
opt = Flux.Optimise.ADAM()
Flux.train!(loss, params(model), data, opt, cb = callback)

V2: put sigmoid inside Dense()

model = Chain(Dense(D, 1, sigmoid) ) 
Flux.train!(loss, params(model), data, opt, cb = callback)
1 Like