# Problems using Flux

**URL:** <https://discourse.julialang.org/t/problems-using-flux/99622>\
**Category:** New to Julia\
**Created:** [May 30, 2023, 6:04pm UTC](https://discourse.julialang.org/t/problems-using-flux/99622 "2023-05-30T18:04:11Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jcverni](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jcverni/32/50682_2.png) [@Jcverni](https://discourse.julialang.org/u/Jcverni)\
**Post date:** [May 30, 2023, 6:04pm UTC](https://discourse.julialang.org/t/problems-using-flux/99622/1 "2023-05-30T18:04:11Z")

</div>

Hi, I’m Juan from Argentina.  
I can’t get the following program to work and I don’t understand what’s wrong, could someone help me or give me a hint.  
This is the code:

using Flux,Plots, Images,MLDatasets  
using Flux: crossentropy, onecold, onehotbatch, train!,params  
using LinearAlgebra, Random,Statistics

Random.seed!(1)

X\_train\_raw, y\_train\_raw = MLDatasets.MNIST(split=:train)[:]  
x\_test\_raw, y\_test\_raw = MLDatasets.MNIST(split=:test)[:]

X\_train = Flux.flatten(X\_train\_raw)  
x\_test = Flux.flatten(x\_test\_raw)

y\_train = onehotbatch(y\_train\_raw,0:9)  
y\_test = onehotbatch(y\_test\_raw,0:9)

data = [X\_train,y\_train]

###########################################################  
model = Chain(Dense(28\*28=\>32,relu),Dense(32=\>10),softmax)  
###########################################################

loss(x,y) = crossentropy(model(x),y)  
ps = params(model)  
learning\_rate = 0.01  
#opt = Adam(learning\_rate)  
opt = Flux.setup(Adam(learning\_rate), model)  
loss\_history =   
epochs = 500

#for epochs in 1:epochs

```
Flux.train!(loss,model,data,opt)
train_loss = loss(X_train,y_train)
push!(loss_history,train_loss)
println("Epoch = $epoch : Trainin_loss = $train_loss")

```

#end

Thanks

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [May 30, 2023, 6:33pm UTC](https://discourse.julialang.org/t/problems-using-flux/99622/2 "2023-05-30T18:33:26Z")

</div>

Could you explain what you think is wrong?

---

<div class="post-metadata">

**Author:** ![Jcverni](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jcverni/32/50682_2.png) [@Jcverni](https://discourse.julialang.org/u/Jcverni)\
**Post date:** [May 30, 2023, 7:07pm UTC](https://discourse.julialang.org/t/problems-using-flux/99622/3 "2023-05-30T19:07:03Z")

</div>

Hi, this is the error:

ERROR: MethodError: no method matching (::Dense{typeof(relu), Matrix{Float32}, Vector{Float32}})(::Chain{Tuple{Dense{typeof(relu), Matrix{Float32}, Vector{Float32}}, Dense{typeof(identity), Matrix{Float32}, Vector{Float32}}, typeof(softmax)}})

Closest candidates are:  
(::Dense)(::AbstractVecOrMat)  
@ Flux ~/.julia/packages/Flux/EHgZm/src/layers/basic.jl:170  
(::Dense)(::AbstractArray)  
@ Flux ~/.julia/packages/Flux/EHgZm/src/layers/basic.jl:177

Stacktrace:  
[1] macro expansion  
@ ~/.julia/packages/Zygote/HTsWj/src/compiler/interface2.jl:101 [inlined]  
[2] \_pullback(ctx::Zygote.Context{false}, f::Dense{typeof(relu), Matrix{Float32}, Vector{Float32}}, args::Chain{Tuple{Dense{typeof(relu), Matrix{Float32}, Vector{Float32}}, Dense{typeof(identity), Matrix{Float32}, Vector{Float32}}, typeof(softmax)}})  
@ Zygote ~/.julia/packages/Zygote/HTsWj/src/compiler/interface2.jl:101  
[3] macro expansion  
@ ~/.julia/packages/Flux/EHgZm/src/layers/basic.jl:53 [inlined]  
[4] \_pullback  
@ ~/.julia/packages/Flux/EHgZm/src/layers/basic.jl:53 [inlined]  
[5] \_pullback(::Zygote.Context{false}, ::typeof(Flux.\_applychain), ::Tuple{Dense{typeof(relu), Matrix{Float32}, Vector{Float32}}, Dense{typeof(identity), Matrix{Float32}, Vector{Float32}}, typeof(softmax)}, ::Chain{Tuple{Dense{typeof(relu), Matrix{Float32}, Vector{Float32}}, Dense{typeof(identity), Matrix{Float32}, Vector{Float32}}, typeof(softmax)}})  
@ Zygote ~/.julia/packages/Zygote/HTsWj/src/compiler/interface2.jl:0  
[6] \_pullback  
@ ~/.julia/packages/Flux/EHgZm/src/layers/basic.jl:51 [inlined]  
[7] \_pullback(ctx::Zygote.Context{false}, f::Chain{Tuple{Dense{typeof(relu), Matrix{Float32}, Vector{Float32}}, Dense{typeof(identity), Matrix{Float32}, Vector{Float32}}, typeof(softmax)}}, args::Chain{Tuple{Dense{typeof(relu), Matrix{Float32}, Vector{Float32}}, Dense{typeof(identity), Matrix{Float32}, Vector{Float32}}, typeof(softmax)}})  
@ Zygote ~/.julia/packages/Zygote/HTsWj/src/compiler/interface2.jl:0  
[8] \_pullback  
@ ~/Documents/Julia/Julia/Pruebas.jl:25 [inlined]  
[9] \_pullback(::Zygote.Context{false}, ::typeof(loss), ::Chain{Tuple{Dense{typeof(relu), Matrix{Float32}, Vector{Float32}}, Dense{typeof(identity), Matrix{Float32}, Vector{Float32}}, typeof(softmax)}}, ::Matrix{Float32})  
@ Zygote ~/.julia/packages/Zygote/HTsWj/src/compiler/interface2.jl:0  
[10] \_apply  
@ ./boot.jl:838 [inlined]  
[11] adjoint  
@ ~/.julia/packages/Zygote/HTsWj/src/lib/lib.jl:203 [inlined]  
[12] \_pullback  
@ ~/.julia/packages/ZygoteRules/OgCVT/src/adjoint.jl:66 [inlined]  
[13] \_pullback  
@ ~/.julia/packages/Flux/EHgZm/src/train.jl:107 [inlined]  
[14] \_pullback(ctx::Zygote.Context{false}, f::Flux.Train.var"#4#5"{typeof(loss), Tuple{Matrix{Float32}}}, args::Chain{Tuple{Dense{typeof(relu), Matrix{Float32}, Vector{Float32}}, Dense{typeof(identity), Matrix{Float32}, Vector{Float32}}, typeof(softmax)}})  
@ Zygote ~/.julia/packages/Zygote/HTsWj/src/compiler/interface2.jl:0  
[15] pullback(f::Function, cx::Zygote.Context{false}, args::Chain{Tuple{Dense{typeof(relu), Matrix{Float32}, Vector{Float32}}, Dense{typeof(identity), Matrix{Float32}, Vector{Float32}}, typeof(softmax)}})  
@ Zygote ~/.julia/packages/Zygote/HTsWj/src/compiler/interface.jl:44  
[16] pullback  
@ ~/.julia/packages/Zygote/HTsWj/src/compiler/interface.jl:42 [inlined]  
[17] withgradient(f::Function, args::Chain{Tuple{Dense{typeof(relu), Matrix{Float32}, Vector{Float32}}, Dense{typeof(identity), Matrix{Float32}, Vector{Float32}}, typeof(softmax)}})  
@ Zygote ~/.julia/packages/Zygote/HTsWj/src/compiler/interface.jl:132  
[18] macro expansion  
@ ~/.julia/packages/Flux/EHgZm/src/train.jl:107 [inlined]  
[19] macro expansion  
@ ~/.julia/packages/ProgressLogging/6KXlp/src/ProgressLogging.jl:328 [inlined]  
[20] train!(loss::Function, model::Chain{Tuple{Dense{typeof(relu), Matrix{Float32}, Vector{Float32}}, Dense{typeof(identity), Matrix{Float32}, Vector{Float32}}, typeof(softmax)}}, data::Vector{AbstractMatrix}, opt::NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Optimisers.Leaf{Optimisers.Adam{Float64}, Tuple{Matrix{Float32}, Matrix{Float32}, Tuple{Float64, Float64}}}, Optimisers.Leaf{Optimisers.Adam{Float64}, Tuple{Vector{Float32}, Vector{Float32}, Tuple{Float64, Float64}}}, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Optimisers.Leaf{Optimisers.Adam{Float64}, Tuple{Matrix{Float32}, Matrix{Float32}, Tuple{Float64, Float64}}}, Optimisers.Leaf{Optimisers.Adam{Float64}, Tuple{Vector{Float32}, Vector{Float32}, Tuple{Float64, Float64}}}, Tuple{}}}, Tuple{}}}}; cb::Nothing)  
@ Flux.Train ~/.julia/packages/Flux/EHgZm/src/train.jl:105  
[21] train!(loss::Function, model::Chain{Tuple{Dense{typeof(relu), Matrix{Float32}, Vector{Float32}}, Dense{typeof(identity), Matrix{Float32}, Vector{Float32}}, typeof(softmax)}}, data::Vector{AbstractMatrix}, opt::NamedTuple{(:layers,), Tuple{Tuple{NamedTuple{(:weight, :bias, :σ), Tuple{Optimisers.Leaf{Optimisers.Adam{Float64}, Tuple{Matrix{Float32}, Matrix{Float32}, Tuple{Float64, Float64}}}, Optimisers.Leaf{Optimisers.Adam{Float64}, Tuple{Vector{Float32}, Vector{Float32}, Tuple{Float64, Float64}}}, Tuple{}}}, NamedTuple{(:weight, :bias, :σ), Tuple{Optimisers.Leaf{Optimisers.Adam{Float64}, Tuple{Matrix{Float32}, Matrix{Float32}, Tuple{Float64, Float64}}}, Optimisers.Leaf{Optimisers.Adam{Float64}, Tuple{Vector{Float32}, Vector{Float32}, Tuple{Float64, Float64}}}, Tuple{}}}, Tuple{}}}})  
@ Flux.Train ~/.julia/packages/Flux/EHgZm/src/train.jl:102  
[22] top-level scope  
@ ~/Documents/Julia/Julia/Pruebas.jl

---

<div class="post-metadata">

**Author:** ![Tomas\_Pevny](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tomas_pevny/32/25466_2.png) [@Tomas\_Pevny](https://discourse.julialang.org/u/Tomas_Pevny)\
**Post date:** [May 30, 2023, 7:32pm UTC](https://discourse.julialang.org/t/problems-using-flux/99622/4 "2023-05-30T19:32:15Z")

</div>

So the problem is twofold.

First, the loss function should be defined as `loss(model, x,y) = crossentropy(model(x),y)` **and not** with an implicit `model` as `loss(x,y) = crossentropy(model(x),y)` (the latter assumes differentiation with implicit parameters, which is now abandoned.

The second problem is that data samples should be `Vector` (or anything iterable with `enumerate`) of tuples and not of vectors. So whereas this is correct `data = [(X_train,y_train)]`, this is wrong `data = [X_train,y_train]`.

A minor comment is that I am not sure if you want to do a full batch, which is what you do now.

Below find a fixed code, where I have frankly not bothered with minibatching.

```julia
using Flux, MLDatasets
using Flux: crossentropy, onecold, onehotbatch, train!,params
using LinearAlgebra, Random, Statistics

Random.seed!(1)

x_train_raw, y_train_raw = MLDatasets.MNIST(split=:train)[:]
x_test_raw, y_test_raw = MLDatasets.MNIST(split=:test)[:]

x_train = Flux.flatten(x_train_raw)
x_test = Flux.flatten(x_test_raw)

y_train = onehotbatch(y_train_raw,0:9)
y_test = onehotbatch(y_test_raw,0:9)

###########################################################
model = Chain(Dense(28*28=>32,relu),Dense(32=>10),softmax)
###########################################################

loss(model, x,y) = crossentropy(model(x),y)
learning_rate = 0.01
opt = Flux.setup(Adam(learning_rate), model)

data = [(x_train[:,1:100],y_train[:,1:100])]
train!(loss, model, data, opt)

```

---

<div class="post-metadata">

**Author:** ![Jcverni](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jcverni/32/50682_2.png) [@Jcverni](https://discourse.julialang.org/u/Jcverni)\
**Post date:** [May 30, 2023, 8:38pm UTC](https://discourse.julialang.org/t/problems-using-flux/99622/5 "2023-05-30T20:38:30Z")

</div>

Thanks for your concern, but it keeps giving me the same error:

ERROR: MethodError: no method matching (::Dense{typeof(relu), Matrix{Float32}, Vector{Float32}})(::Chain{Tuple{Dense{typeof(relu), Matrix{Float32}, Vector{Float32}}, Dense{typeof(identity), Matrix{Float32}, Vector{Float32}}, typeof(softmax)}})

Closest candidates are:  
(::Dense)(::AbstractVecOrMat)  
@ Flux ~/.julia/packages/Flux/EHgZm/src/layers/basic.jl:170  
(::Dense)(::AbstractArray)  
@ Flux ~/.julia/packages/Flux/EHgZm/src/layers/basic.jl:177

---

<div class="post-metadata">

**Author:** ![mcabbott](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mcabbott/32/6603_2.png) [@mcabbott](https://discourse.julialang.org/u/mcabbott)\
**Post date:** [May 31, 2023, 3:45am UTC](https://discourse.julialang.org/t/problems-using-flux/99622/6 "2023-05-31T03:45:22Z")

</div>

> [@Tomas\_Pevny](#):
>
> `data = [[x_train[:,1:100],y_train[:,1:100]]]`

I think this won’t work, due to a quirk of `train!` that sometimes it splats elements of `data`, but not always – the last sentence here:

```julia
julia> data = [(x_train[:,1:100], y_train[:,1:100])];

julia> train!(loss, model, data, opt)

help?> train!
search: trainmode! y_train x_train y_train_raw x_train_raw trailing_ones trailing_zeros withgradient

  train!(loss, model, data, opt_state)

  Uses a loss function and training data to improve the model's parameters according to a
  particular optimisation rule encoded in opt_state. Iterates through data once, evaluating for
  each d in data either loss(model, d...) if d isa Tuple, or else loss(model, d) for other d.

```

---

<div class="post-metadata">

**Author:** ![Tomas\_Pevny](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tomas_pevny/32/25466_2.png) [@Tomas\_Pevny](https://discourse.julialang.org/u/Tomas_Pevny)\
**Post date:** [May 31, 2023, 6:28am UTC](https://discourse.julialang.org/t/problems-using-flux/99622/7 "2023-05-31T06:28:01Z")

</div>

@mcabbott is right, I had a typo there, where I have not fixed the problem of how data should be represented. I have fixed the example and it should be right now. @Jcverni Can you check it again?  
For completness, I tested the script with julia 1.9.0 and the package versions are

```julia
Status `/private/tmp/Project.toml`
  [587475ba] Flux v0.13.16
  [eb30cadb] MLDatasets v0.7.9

```

---

<div class="post-metadata">

**Author:** ![Jcverni](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jcverni/32/50682_2.png) [@Jcverni](https://discourse.julialang.org/u/Jcverni)\
**Post date:** [June 6, 2023, 2:48am UTC](https://discourse.julialang.org/t/problems-using-flux/99622/8 "2023-06-06T02:48:33Z")

</div>

Thank you so much. It worked!!