# Simple custom recurrent layer in Flux

**URL:** <https://discourse.julialang.org/t/simple-custom-recurrent-layer-in-flux/26740>\
**Category:** Machine Learning\
**Tags:** flux\
**Created:** [July 24, 2019, 2:14pm UTC](https://discourse.julialang.org/t/simple-custom-recurrent-layer-in-flux/26740 "2019-07-24T14:14:45Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Rassibassi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rassibassi/32/9697_2.png) [@Rassibassi](https://discourse.julialang.org/u/Rassibassi)\
**Post date:** [July 24, 2019, 2:14pm UTC](https://discourse.julialang.org/t/simple-custom-recurrent-layer-in-flux/26740/1 "2019-07-24T14:14:45Z")

</div>

Hi,

I am looking into Flux and trying to understand the recurrent functionality. The following code defines a first order low pass filter with `Flux.Recur`.

```julia
using Flux
using Flux.Tracker

c = 0.5

function LP(h, x)
    h = c * x + (1.0 - c) * h
    return h, h
end

h = 0.
recurLP = Flux.Recur(LP, h)

function model(x)
    out = zeros(1000,1)
    for ii = 1:1000
        out[ii] = recurLP(x[ii])
    end
    return out
end

function loss(x, y)    
    return sum((model(x) .- y).^2) 
end

```

Let’s filter some noise and calculate the loss.

```julia
inp = randn(1000,1);
loss(inp, inp)

```

317.4691466179187

It seems that the filter is filtering something 😉  
So far so good, now we could optimize the filter coefficient, such that the filter stops filtering, e.g. for `c = 1.0` the loss should become 0.  
However, neither:

```julia
Flux.train!(loss, params(model), [(inp, inp)], ADAM(0.1))

```

nor

```julia
c = param(c)
gs = Tracker.gradient(() -> loss(inp, inp), params(c))

```

seem to work.

I get error messages  
`MethodError: no method matching back!(::Float64)`  
and  
`MethodError: no method matching Float64(::Tracker.TrackedReal{Float64})`, respectively.

What am I missing?

Thanks in advance 🙂

---

<div class="post-metadata">

**Author:** ![merckxiaan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/merckxiaan/32/4697_2.png) [@merckxiaan](https://discourse.julialang.org/u/merckxiaan)\
**Post date:** [July 28, 2019, 3:23pm UTC](https://discourse.julialang.org/t/simple-custom-recurrent-layer-in-flux/26740/2 "2019-07-28T15:23:10Z")

</div>

The problem is that your loss is a regular Float64 instead of a Trackedreal number. Through this, Tracker is unable to backpropagate.  
Also, I believe mutation of arrays will give difficulties so it’s better to `push!` to the `out` array.

```julia
function model(x)
    out = []
    for feature in x
        push!(out, recurLP(feature))
    end
    return out
end

```

---

<div class="post-metadata">

**Author:** ![Rassibassi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rassibassi/32/9697_2.png) [@Rassibassi](https://discourse.julialang.org/u/Rassibassi)\
**Post date:** [September 27, 2019, 10:45am UTC](https://discourse.julialang.org/t/simple-custom-recurrent-layer-in-flux/26740/3 "2019-09-27T10:45:36Z")

</div>

Hej,  
Thanks for the help. I had another look and now made it work. Instead of looping over the timeseries we can also use the dot notation. And also the problem with Tracker is solved. Here the code:

```julia
using Flux
using Flux.Tracker

c = 0.5

function LP(h, x)
    h = c * x + (1.0 - c) * h
    return h, h
end

h = 0.
recurLP = Flux.Recur(LP, h)

function model(x)
    return recurLP.(x)
end

function loss(x, y)    
    return sum((model(x) .- y).^2) 
end

inp = randn(1000,1);
println(loss(inp, inp))

Flux.reset!(recurLP)
println(recurLP.state)

c = param(c)
gs = Tracker.gradient(() -> loss(inp, inp), params(c))
Δ = gs[c]

for i = 1:50
    Tracker.update!(c, -0.00001Δ)
    if i%10 == 0
        println("loss= ", loss(inp, inp))
        println("c= ", c)
        println()
    end
end

```