# MethodError: no method matching Adam(::Float64)

**URL:** <https://discourse.julialang.org/t/methoderror-no-method-matching-adam-float64/112009>\
**Category:** General Usage\
**Created:** [March 23, 2024, 8:50am UTC](https://discourse.julialang.org/t/methoderror-no-method-matching-adam-float64/112009 "2024-03-23T08:50:16Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![sharmaviven1](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sharmaviven1/32/207486_2.png) [@sharmaviven1](https://discourse.julialang.org/u/sharmaviven1)\
**Post date:** [March 23, 2024, 8:50am UTC](https://discourse.julialang.org/t/methoderror-no-method-matching-adam-float64/112009/1 "2024-03-23T08:50:16Z")

</div>

Hello all,

I have been trying to setup NeuralODE to learn the behavior of a stiff ODE but in the process I am running into this strange error. I have tried using all Float32 values only but it seems that there is some issue with the implementation.

I am new to Julia and am trying to learn so any help would be greatly appreciated.

Thank you  
Here is the code:

```julia
using ModelingToolkit, DifferentialEquations, Plots, Lux, Optimization, OptimizationOptimJL, Random, Plots, DiffEqFlux, ComponentArrays 
using ModelingToolkit: t_nounits as t, D_nounits as D
#
params = @parameters begin
    R = 8.31, [description = "Gas constant, in J/(K.mol)"]
    T = 1573, [description = "temperature, K"]
    Q = 453_000, [description = "heat of..., J/mol"]
    M = 135e3, [description = "..., in MPa"]
    R°_p = 10^(-0.95), [description = "..., 1/(MPa^5.s)"]
    R°_gb = 10^3.53, [description = "..., 1/(MPa^4.s)"]
    A° = 10^6.94, [description = "..., 1/(MPa^2.s)"]
    d = 400e-6, [description = "grain size, m"]
    β = 2, [description = "..., ..."]
    μ = 65e3, [description = "..., MPa"]
    b = 5e-10, [description = "..., m"]
    Exp = exp(-Q/(R*T))
    #
    σ_T_max = Float32(1.8e3)
    σ_d = β*μ*b/d
    σ_ref = Float32(3.1e3) # σ∗_p*R*T/Q
    σ = 100
end
#
vars = @variables begin
    σ_T(t) = 1
    ϵ°(t) # = A°*Exp*1^2*sinh((σ - σ_T - σ_d)/σ_ref)
end
#
eqs = [ϵ° ~ A°*Exp*σ_T^2*sinh((σ - σ_T - σ_d)/σ_ref),
        D(σ_T) ~ M*( ϵ°*(σ_T + σ_d)/σ_T - abs(ϵ°)*σ_T/σ_T_max - R°_p*Exp*σ_T^5 - R°_gb*Exp*σ_T^3*σ_d )]

@named sys_eq = ODESystem(eqs,t,vars,params)
sys = structural_simplify(sys_eq)

tspan = (0.0,20000.0)
datasize = 50
tsteps = range(tspan[1], tspan[2], length = datasize)
prob = ODEProblem(sys,[],tspan)

sol_array = Array(solve(prob,TRBDF2(),saveat=tsteps))
sol_array = Float32.(sol_array)
#sol=solve(prob,TRBDF2(),saveat=tsteps)

plot(sol, lw=2.5, lc=:blue,legend=:topleft, xlabel="time in sec",ylabel="Stress in MPa")

# For saving solution outputs
using DataFrames
df = DataFrame(sol)
using CSV
CSV.write("sol_values.csv", df)

# shaping information for the NN
math_law(x) = 2.0.*x.^5-1.0.*x.^3 # Just some approximate form of the eqn
# Construct the layer
dudt2 = Lux.Chain(x -> math_law(x),Lux.Dense(1, 50, tanh),Lux.Dense(50, 1))
rng = Random.default_rng()
# Initialization

p, st = Lux.setup(rng, dudt2)

# _______________________________________________________________________________
prob_neuralode = NeuralODE(dudt2, tspan, Tsit5(), saveat = tsteps)
# ______________________________________________________________________________
function predict_neuralode(p)
  Array(prob_neuralode(u0, p, st)[1])
end
# ______________________________________________________________________________
function loss_neuralode(p)
    pred = predict_neuralode(p)
    loss = sum(abs2.(sol_array .- pred))
    return loss, pred
end
# ______________________________________________________________________________
callback = function (p, l, pred; doplot = false)
    # plot current prediction against data
    if doplot
      plt = Plots.scatter(tsteps, sol_array[1,:], label = "data")  
      Plots.scatter!(plt, tsteps, pred[1,:], label = "prediction")
      display(plot(plt))
    end
    return false
end
# ______________________________________________________________________________
pinit = ComponentArray(p)
# Initial condition
u0 = Float32[1.0]
callback(pinit, loss_neuralode(pinit)...; doplot=true)
# ______________________________________________________________________________
# use Optimization.jl to solve the problem
adtype = Optimization.AutoZygote()
optf = Optimization.OptimizationFunction((x, p) -> loss_neuralode(x), adtype)
optprob = Optimization.OptimizationProblem(optf, pinit)
result_neuralode = Optimization.solve(optprob,Adam(0.05),callback = callback,maxiters = 100)
# Retrain using the LBFGS optimizer
optprob2 = remake(optprob,u0 = result_neuralode.u)
result_neuralode2 = Optimization.solve(optprob2,Optim.BFGS(initial_stepnorm=0.01),callback=callback,allow_f_increases = false)
callback(result_neuralode2.u, loss_neuralode(result_neuralode2.u)...; doplot=true)
# _______________________

prob_neuralode2 = NeuralODE(dudt2, tspan2, Tsit5(), saveat = tsteps)
function predict_neuralode(p)
    Array(prob_neuralode2(forecastu0, p, st)[1])
  end

```

---

<div class="post-metadata">

**Author:** ![oheil](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oheil/32/220745_2.png) [@oheil](https://discourse.julialang.org/u/oheil)\
**Post date:** [March 23, 2024, 9:36am UTC](https://discourse.julialang.org/t/methoderror-no-method-matching-adam-float64/112009/2 "2024-03-23T09:36:36Z")

</div>

Have a look here: [Optimizer not found error](https://discourse.julialang.org/t/optimizer-not-found-error/92364)

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [March 23, 2024, 7:30pm UTC](https://discourse.julialang.org/t/methoderror-no-method-matching-adam-float64/112009/3 "2024-03-23T19:30:03Z")

</div>

In late 2023 Optim.jl had a release that added an `Adam` algorithm, and to the dismay of everyone in the community, it used exactly the same name as the Optimisers.jl `Adam` algorithm. Both export `Adam`, so what you’re leaving off here is that you received a warning in your REPL that said you need to disambiguate which `Adam` you’re referring to.

In other words, you need to change:

```julia
result_neuralode = Optimization.solve(optprob,Adam(0.05),callback = callback,maxiters = 100)

```

to:

```julia
result_neuralode = Optimization.solve(optprob,OptimizationOptimisers.Adam(0.05),callback = callback,maxiters = 100)

```

to make it explicit.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [March 23, 2024, 7:31pm UTC](https://discourse.julialang.org/t/methoderror-no-method-matching-adam-float64/112009/4 "2024-03-23T19:31:22Z")

</div>

I believe you’re trying to do something similar to this blog post:

> **[Weather forecasting with neural ODEs · DiffEqFlux.jl](https://docs.sciml.ai/DiffEqFlux/dev/examples/neural_ode_weather_forecast/)**
>
> Documentation for DiffEqFlux.jl.

so it may serve as a better guide than whatever older example you’re looking at.

---

<div class="post-metadata">

**Author:** ![sharmaviven1](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sharmaviven1/32/207486_2.png) [@sharmaviven1](https://discourse.julialang.org/u/sharmaviven1)\
**Post date:** [March 26, 2024, 8:42am UTC](https://discourse.julialang.org/t/methoderror-no-method-matching-adam-float64/112009/5 "2024-03-26T08:42:31Z")

</div>

Thanks @oheil and @ChrisRackauckas the code is running now. However, it seems as if the model is not able to learn.

Any suggestions on how to improving the training process?

Viven

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [March 26, 2024, 1:41pm UTC](https://discourse.julialang.org/t/methoderror-no-method-matching-adam-float64/112009/6 "2024-03-26T13:41:48Z")

</div>

Lower tolerances or learning rate?

---

<div class="post-metadata">

**Author:** ![sharmaviven1](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sharmaviven1/32/207486_2.png) [@sharmaviven1](https://discourse.julialang.org/u/sharmaviven1)\
**Post date:** [March 27, 2024, 7:10am UTC](https://discourse.julialang.org/t/methoderror-no-method-matching-adam-float64/112009/7 "2024-03-27T07:10:38Z")

</div>

It is getting stuck at some local minima (see plot below). All other parameters are same as that in the initial code.

```julia
# shaping information for the NN
math_law(x) = 1000.0*x.^5 

```

 ![image](https://global.discourse-cdn.com/julialang/original/3X/2/9/29cd3b662a8dece30c6c03ce7351eccfcc95a654.png)

Thanks  
Viven

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [March 27, 2024, 8:08am UTC](https://discourse.julialang.org/t/methoderror-no-method-matching-adam-float64/112009/8 "2024-03-27T08:08:39Z")

</div>

That doesn’t look like a local optima. I’d look for other issues, like saturating loss functions.