# Issues with recurrent layers in Flux.jl

**URL:** <https://discourse.julialang.org/t/issues-with-recurrent-layers-in-flux-jl/67815>\
**Category:** Machine Learning\
**Tags:** flux\
**Created:** [September 7, 2021, 1:03pm UTC](https://discourse.julialang.org/t/issues-with-recurrent-layers-in-flux-jl/67815 "2021-09-07T13:03:23Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![shindler25](https://avatars.discourse-cdn.com/v4/letter/s/6de8d8/32.png) [@shindler25](https://discourse.julialang.org/u/shindler25)\
**Post date:** [September 7, 2021, 1:03pm UTC](https://discourse.julialang.org/t/issues-with-recurrent-layers-in-flux-jl/67815/1 "2021-09-07T13:03:23Z")

</div>

_I am using Julia 1.6 and Flux v0.12.6_

I am having issues using the recurrent layers LSTM and RNN in Flux.jl and it does not make sense why this is happening. hopefully someone can see where it is going wrong?

I define the model as:

```julia
julia> my_model = LSTM(1,1)
Recur(
  LSTMCell(1, 1), # 14 parameters
) # Total: 5 trainable arrays, 14 parameters,
          # plus 2 non-trainable, 2 parameters, summarysize 384 bytes.

```

I test it with

```julia
julia> eval = my_model.([1,2,3])

```

And it returns the error

```julia
ERROR: MethodError: no method matching (::Flux.LSTMCell{Matrix{Float32}, Vector{Float32}, Tuple{Matrix{Float32}, Matrix{Float32}}})(::Tuple{Matrix{Float32}, Matrix{Float32}}, ::Int64)
Closest candidates are:
  (::Flux.LSTMCell{A, V, var"#s318"} where var"#s318"<:Tuple{AbstractMatrix{T}, AbstractMatrix{T}})(::Any, ::Union{AbstractVector{T}, AbstractMatrix{T}, Flux.OneHotArray}) where {A, V, T} at /home/shindler/.julia/packages/Flux/Zz9RI/src/layers/recurrent.jl:137
Stacktrace:
 [1] (::Flux.Recur{Flux.LSTMCell{Matrix{Float32}, Vector{Float32}, Tuple{Matrix{Float32}, Matrix{Float32}}}, Tuple{Matrix{Float32}, Matrix{Float32}}})(x::Int64)
   @ Flux ~/.julia/packages/Flux/Zz9RI/src/layers/recurrent.jl:34
 [2] _broadcast_getindex_evalf
   @ ./broadcast.jl:648 [inlined]
 [3] _broadcast_getindex
   @ ./broadcast.jl:621 [inlined]
 [4] getindex
   @ ./broadcast.jl:575 [inlined]
 [5] copy
   @ ./broadcast.jl:922 [inlined]
 [6] materialize(bc::Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{1}, Nothing, Flux.Recur{Flux.LSTMCell{Matrix{Float32}, Vector{Float32}, Tuple{Matrix{Float32}, Matrix{Float32}}}, Tuple{Matrix{Float32}, Matrix{Float32}}}, Tuple{Vector{Int64}}})
   @ Base.Broadcast ./broadcast.jl:883
 [7] top-level scope
   @ REPL[31]:1
 [8] top-level scope
   @ ~/.julia/packages/CUDA/9T5Sq/src/initialization.jl:66

```

I get the same issue when I run the model without the `.`

```julia
julia> my_model = LSTM(3,1)
Recur(LSTMCell(3, 1))

julia> my_model([3,2,1])
ERROR: MethodError: no method matching (::Flux.LSTMCell{Matrix{Float32}, Vector{Float32}, Tuple{Matrix{Float32}, Matrix{Float32}}})(::Tuple{Matrix{Float32}, Matrix{Float32}}, ::Vector{Int64})
Closest candidates are:
  (::Flux.LSTMCell{A, V, var"#s268"} where var"#s268"<:Tuple{AbstractMatrix{T}, AbstractMatrix{T}})(::Any, ::Union{AbstractVector{T}, AbstractMatrix{T}, Flux.OneHotArray}) where {A, V, T} at /home/shindler/.julia/packages/Flux/0c9kI/src/layers/recurrent.jl:137
Stacktrace:
 [1] (::Flux.Recur{Flux.LSTMCell{Matrix{Float32}, Vector{Float32}, Tuple{Matrix{Float32}, Matrix{Float32}}}, Tuple{Matrix{Float32}, Matrix{Float32}}})(x::Vector{Int64})
   @ Flux ~/.julia/packages/Flux/0c9kI/src/layers/recurrent.jl:34
 [2] top-level scope
   @ REPL[40]:1
 [3] top-level scope
   @ ~/.julia/packages/CUDA/fRSUT/src/initialization.jl:52

```

I have the issue with a RNN as well

```julia
julia> my_model = RNN(3,1)
Recur(RNNCell(3, 1, tanh))

julia> my_model([3,2,1])
ERROR: MethodError: no method matching (::Flux.RNNCell{typeof(tanh), Matrix{Float32}, Vector{Float32}, Matrix{Float32}})(::Matrix{Float32}, ::Vector{Int64})
Closest candidates are:
  (::Flux.RNNCell{F, A, V, var"#s269"} where var"#s269"<:AbstractMatrix{T})(::Any, ::Union{AbstractVector{T}, AbstractMatrix{T}, Flux.OneHotArray}) where {F, A, V, T} at /home/shindler/.julia/packages/Flux/0c9kI/src/layers/recurrent.jl:83
Stacktrace:
 [1] (::Flux.Recur{Flux.RNNCell{typeof(tanh), Matrix{Float32}, Vector{Float32}, Matrix{Float32}}, Matrix{Float32}})(x::Vector{Int64})
   @ Flux ~/.julia/packages/Flux/0c9kI/src/layers/recurrent.jl:34
 [2] top-level scope
   @ REPL[42]:1
 [3] top-level scope
   @ ~/.julia/packages/CUDA/fRSUT/src/initialization.jl:52

```

But a Dense layer seems to work fine

```julia
julia> my_model = Dense(3,1)
Dense(3, 1)

julia> my_model([3,2,1])
1-element Vector{Float32}:
 -1.0215458

```

---

<div class="post-metadata">

**Author:** ![ToucheSir](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/touchesir/32/14411_2.png) [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)\
**Post date:** [September 7, 2021, 3:58pm UTC](https://discourse.julialang.org/t/issues-with-recurrent-layers-in-flux-jl/67815/2 "2021-09-07T15:58:28Z")

</div>

`[3,2,1]` is a `Vector{Int64}`, and the basic Flux layers are only guaranteed to work with floating point types. That you can pass an int array into `Dense` is a happy coincidence, and even then not guaranteed to work in perpetuity. Try `Float32[3,2,1]` or allocating a float array by any other means instead, and the RNNs should work as expected.

As for this:

> [@shindler25](#):
>
> I test it with
> 
> ```julia
> julia> eval = my_model.([1,2,3])
> 
> ```

This is probably not doing what you want or expect of it. Broadcasting over an array applies the broadcasted function element-wise, so this is equivalent to `[my_model(1), my_model(2), my_model(3)]`. See [Recurrence · Flux](https://fluxml.ai/Flux.jl/stable/models/recurrence/#Working-with-sequences) for a guide on how to properly use sequences with Flux RNNs.

---

<div class="post-metadata">

**Author:** ![shindler25](https://avatars.discourse-cdn.com/v4/letter/s/6de8d8/32.png) [@shindler25](https://discourse.julialang.org/u/shindler25)\
**Post date:** [September 7, 2021, 6:01pm UTC](https://discourse.julialang.org/t/issues-with-recurrent-layers-in-flux-jl/67815/3 "2021-09-07T18:01:51Z")

</div>

Thank you,

Using Float32 indeed solves all my problems.

Extra thanks for pointing out the broadcasts issue. I somehow missed that. (Need to be more careful about trusting random internet blogs).

---

<div class="post-metadata">

**Author:** ![ToucheSir](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/touchesir/32/14411_2.png) [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)\
**Post date:** [September 8, 2021, 4:45pm UTC](https://discourse.julialang.org/t/issues-with-recurrent-layers-in-flux-jl/67815/4 "2021-09-08T16:45:26Z")

</div>

Do you mind posting the blog post in question? It might be worth someone suggesting a change to add a disclaimer or to help the author rewrite the post.

---

<div class="post-metadata">

**Author:** ![shindler25](https://avatars.discourse-cdn.com/v4/letter/s/6de8d8/32.png) [@shindler25](https://discourse.julialang.org/u/shindler25)\
**Post date:** [September 9, 2021, 12:56pm UTC](https://discourse.julialang.org/t/issues-with-recurrent-layers-in-flux-jl/67815/5 "2021-09-09T12:56:39Z")

</div>

The specific post I referred to is:

> **[A Basic RNN](https://learningjulia.com/2019/10/11/a-basic-rnn.html)**
>
> In the epic adventure of audio processing and neural networks with Julia, the theme of today is RNNs! I’ll walk through how I built my first basic RNN in Julia with Flux.jl. This is all in pursuit of the Trebekian project.

> Now to actually evaluate the model on a sequence of inputs, you have to call it like this:
> 
> `julia> simple_rnn.([1, 2, 3])`
> 
> Notice the dot notation here - since our RNN only takes one input at a time, we need to apply the RNN on the sequence of inputs we provide one at a time. Then, if we take the last element of the output (after it has seen the entire sequence) we expect to see the sum of all the inputs.

But I have seen broadcasts used fairly frequently in the context of time series NNs in Julia.

Most notably: [https://github.com/FluxML/model-zoo/blob/master/text/char-rnn/char-rnn.jl](https://github.com/FluxML/model-zoo/blob/master/text/char-rnn/char-rnn.jl)

> ```julia
> function loss(xs, ys)
> l = sum(logitcrossentropy.(m.(xs), ys))
> return l
> end
> 
> ```

---

<div class="post-metadata">

**Author:** ![albheim](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/albheim/32/34660_2.png) [@albheim](https://discourse.julialang.org/u/albheim)\
**Post date:** [September 13, 2021, 12:16pm UTC](https://discourse.julialang.org/t/issues-with-recurrent-layers-in-flux-jl/67815/6 "2021-09-13T12:16:15Z")

</div>

But I think in both those cases it is actually warranted to use broadcasting.

In the first case they mention that we want to apply the RNN on one input at a time since the RNN only take one input at a time, but updates its internal state each time so you do want to call it on each individual input in order. This is what would happen when you broadcast.

In the second case it seems the data are sequences of one-hot encodings of characters, so the RNN should be run on the one-hot vector of each individual character in a batch which I believe is what they achieve with the broadcast (haven’t tried the code myself but that is my understanding of the code).

---

<div class="post-metadata">

**Author:** ![shindler25](https://avatars.discourse-cdn.com/v4/letter/s/6de8d8/32.png) [@shindler25](https://discourse.julialang.org/u/shindler25)\
**Post date:** [September 13, 2021, 12:41pm UTC](https://discourse.julialang.org/t/issues-with-recurrent-layers-in-flux-jl/67815/7 "2021-09-13T12:41:07Z")

</div>

Obviously, I have a limited understanding of this 🙃. But based on the documentation, the issue is not processing one input at a time, rather the use of broadcasts to do it.

The [documentation for recurrence](https://fluxml.ai/Flux.jl/stable/models/recurrence/#Working-with-sequences) warns:

> Mapping and broadcasting operations with stateful layers such are discouraged, since the julia language doesn’t guarantee a specific execution order. Therefore, avoid
> 
> ```julia
> y = m.(x)
> # or 
> y = map(m, x)
> 
> ```
> 
> and use explicit loops
> 
> ```julia
> y = [m(x) for x in x]
> 
> ```

So my understanding is that if the order that the data is processed matters then broadcasting shouldn’t be used. 🤷‍♂️

---

<div class="post-metadata">

**Author:** ![albheim](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/albheim/32/34660_2.png) [@albheim](https://discourse.julialang.org/u/albheim)\
**Post date:** [September 13, 2021, 1:17pm UTC](https://discourse.julialang.org/t/issues-with-recurrent-layers-in-flux-jl/67815/8 "2021-09-13T13:17:25Z")

</div>

You are correct, and that makes sense that a loop guarantees the order while a broadcast does not.

Didn’t read everything properly and thought the mixup was between the RNN model and your Dense example for some reason. Thought your gripe with the examples in question was that it didn’t work for you dense model. Next time I’ll try to read the text before trying to answer 😅

---

<div class="post-metadata">

**Author:** ![ToucheSir](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/touchesir/32/14411_2.png) [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)\
**Post date:** [September 13, 2021, 4:26pm UTC](https://discourse.julialang.org/t/issues-with-recurrent-layers-in-flux-jl/67815/9 "2021-09-13T16:26:48Z")

</div>

In fairness, many examples (including, as has been pointed out, the model zoo) used to use broadcasting. However, we ran into a number of issues where broadcasting wasn’t strictly running in left-right order. The reason the docs have been updated but the model zoo hasn’t is simply because we don’t have enough time on our hands as a almost 100% volunteer crew 😅. PRs would be very welcome!