# Error in training Deep Equilibrium Models (DEQ)

**URL:** <https://discourse.julialang.org/t/error-in-training-deep-equilibrium-models-deq/87958>\
**Category:** General Usage\
**Tags:** diffeq, flux\
**Created:** [September 29, 2022, 2:16am UTC](https://discourse.julialang.org/t/error-in-training-deep-equilibrium-models-deq/87958 "2022-09-29T02:16:06Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Molox](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/molox/32/35002_2.png) [@Molox](https://discourse.julialang.org/u/Molox)\
**Post date:** [September 29, 2022, 2:16am UTC](https://discourse.julialang.org/t/error-in-training-deep-equilibrium-models-deq/87958/1 "2022-09-29T02:16:06Z")

</div>

I read the [Blog](https://julialang.org/blog/2021/10/DEQ/#full_example_deq_for_learning_mnist_from_scratch) about “Composability in Julia: Implementing Deep Equilibrium Models via Neural ODEs”.

I tried to reproduce the code to train the MNIST dataset, but it returns an error during training:

```julia
ERROR: MethodError: no method matching needs_concrete_A(::Nothing)
Closest candidates are:
  needs_concrete_A(::LinearSolve.AbstractFactorization) at julia\packages\LinearSolve\tyXtB\src\LinearSolve.jl:34        
  needs_concrete_A(::LinearSolve.AbstractKrylovSubspaceMethod) at julia\packages\LinearSolve\tyXtB\src\LinearSolve.jl:35 
  needs_concrete_A(::LinearSolve.AbstractSolveFunction) at julia\packages\LinearSolve\tyXtB\src\LinearSolve.jl:36     

```

I simplified the original code to the following one:

```julia
using Flux, Statistics
using DiffEqSensitivity, OrdinaryDiffEq, SteadyStateDiffEq

struct DeepEquilibriumNetwork{M,P,RE,A,K}
    model::M
    p::P
    re::RE
    args::A
    kwargs::K
end

Flux.@functor DeepEquilibriumNetwork

function DeepEquilibriumNetwork(model, args...; kwargs...)
    p, re = Flux.destructure(model)
    return DeepEquilibriumNetwork(model, p, re, args, kwargs)
end

Flux.trainable(deq::DeepEquilibriumNetwork) = (deq.p,)

function (deq::DeepEquilibriumNetwork)(x::AbstractArray{T},
                                       p = deq.p) where {T}
    z = deq.re(p)(x)
    # Solving the equation f(u) - u = du = 0
    dudt(u, _p, t) = deq.re(_p)(u .+ x) .- u
    ssprob = SteadyStateProblem(ODEProblem(dudt, z, (zero(T), one(T)), p))
    return solve(ssprob, deq.args...; u0 = z, deq.kwargs...).u
end

function Net()
    return Chain(
        DeepEquilibriumNetwork(Chain(Dense(100, 500, tanh), Dense(500, 100)),
                               DynamicSS(Tsit5(), abstol = 1f-2, reltol = 1f-2)),
    )
end

function loss_function()
    return mean(abs, deq(rand(100,1)) .- rand(100,1))
end

deq = Net();
data = Iterators.repeated((), 10);
Flux.train!(loss_function, Flux.params(deq), data, ADAM(0.01))

```

I just found if I modify the DeepEquilibriumNetwork size to 10 times smaller, it will train and won’t give an error…

```julia
function Net()
    return Chain(
        DeepEquilibriumNetwork(Chain(Dense(10, 50, tanh), Dense(50, 10)),
                               DynamicSS(Tsit5(), abstol = 1f-2, reltol = 1f-2)),
    )
end

```

I am using Julia v1.7. Thank you for any help!

---

<div class="post-metadata">

**Author:** ![Tomas\_Pevny](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tomas_pevny/32/25466_2.png) [@Tomas\_Pevny](https://discourse.julialang.org/u/Tomas_Pevny)\
**Post date:** [September 29, 2022, 4:36am UTC](https://discourse.julialang.org/t/error-in-training-deep-equilibrium-models-deq/87958/2 "2022-09-29T04:36:13Z")

</div>

You should provide minimal working example. Without it no-one can help you.

---

<div class="post-metadata">

**Author:** ![avikpal](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/avikpal/32/6550_2.png) [@avikpal](https://discourse.julialang.org/u/avikpal)\
**Post date:** [September 29, 2022, 5:24am UTC](https://discourse.julialang.org/t/error-in-training-deep-equilibrium-models-deq/87958/3 "2022-09-29T05:24:24Z")

</div>

You should use [GitHub - SciML/DeepEquilibriumNetworks.jl: Deep Equilibrium Networks (but faster!!!)](https://github.com/SciML/DeepEquilibriumNetworks.jl) instead of the code in the blog post. Updating the blog post has been on my [todo list](https://github.com/SciML/DeepEquilibriumNetworks.jl/issues/35) for quite some time.

---

<div class="post-metadata">

**Author:** ![Molox](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/molox/32/35002_2.png) [@Molox](https://discourse.julialang.org/u/Molox)\
**Post date:** [September 29, 2022, 5:41am UTC](https://discourse.julialang.org/t/error-in-training-deep-equilibrium-models-deq/87958/4 "2022-09-29T05:41:04Z")

</div>

Thank you for the advice! I just modified it.

---

<div class="post-metadata">

**Author:** ![Molox](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/molox/32/35002_2.png) [@Molox](https://discourse.julialang.org/u/Molox)\
**Post date:** [October 3, 2022, 4:04am UTC](https://discourse.julialang.org/t/error-in-training-deep-equilibrium-models-deq/87958/5 "2022-10-03T04:04:02Z")

</div>

Thank you!

> [@avikpal](#):
>
> [GitHub - SciML/DeepEquilibriumNetworks.jl](https://github.com/SciML/DeepEquilibriumNetworks.jl)

I was able to run the code (with the package) now.