Using GalacticOptim to optimize, cannot understand the error

yewalenikhil65 · November 11, 2021, 5:28am

I have a vector of ODE problems that i want to optimize, and here is first odeproblem from the probs array which contains my ODEProblem. I am using GalacticOptim to optimize, but am encountering following error. Some code is also attached. What could be the issue here ? I did not understand the error.! Let me know if some more explanation is need!

function loss(p)
    loss = sum(abs2, soln .- org_sol)
    return loss, soln
end
callback = function (p, l, pred)
    display(l)
    #plt = plot(pred)
    #display(plt)
    return false
end
# use GalacticOptim.jl to solve the problem
adtype = GalacticOptim.AutoZygote()
optf = GalacticOptim.OptimizationFunction((x, p) -> loss(x), adtype)

julia> probs[1].p   # parameters to be optimized
18-element Vector{Float32}:
 0.3274659
 0.94486237
 1.0
 1.0
 1.0
 1.0
 0.00990099
 0.00990099
 1.0
 1.0
 1.0
 1.0
 1.0
 1.0
 0.051807642
 0.6663984
 1.0
 1.0

julia> soln = Array(solve(probs[1], p = probs[1].p, saveat = 1.0))  # approximated solution
17×551 Matrix{Float32}:
 5.0  4.18527   4.08949   4.12522   4.18547  …  5.63654   5.63654   5.63654   5.63654
 5.0  4.76808   4.56416   4.38125   4.21494     1.39767   1.39767   1.39767   1.39767
 0.0  0.124666  0.142363  0.138717  0.13336     0.151869  0.151869  0.151869  0.15187
 0.0  1.15974   1.42228   1.48669   1.51855     1.95349   1.95349   1.95349   1.95349
 0.0  1.41916   1.53627   1.58009   1.60998     1.95349   1.95349   1.95349   1.95349
 5.0  1.81066   1.79148   1.79129   1.79129  …  1.79129   1.79129   1.79129   1.79129
 5.0  1.81066   1.79148   1.79129   1.79129     1.79129   1.79129   1.79129   1.79129
 0.0  3.18934   3.20852   3.20871   3.20871     3.20871   3.20871   3.20871   3.20871
 5.0  4.76808   4.56416   4.38125   4.21494     1.39767   1.39767   1.39767   1.39767
 0.0  0.8586    1.39199   1.61517   1.70875     1.95349   1.95349   1.95349   1.95349
 5.0  1.81066   1.79148   1.79129   1.79129  …  1.79129   1.79129   1.79129   1.79129
 5.0  1.81066   1.79148   1.79129   1.79129     1.79129   1.79129   1.79129   1.79129
 0.0  3.18934   3.20852   3.20871   3.20871     3.20871   3.20871   3.20871   3.20871
 5.0  1.81066   1.79148   1.79129   1.79129     1.79129   1.79129   1.79129   1.79129
 5.0  1.81066   1.79148   1.79129   1.79129     1.79129   1.79129   1.79129   1.79129
 0.0  3.18934   3.20852   3.20871   3.20871  …  3.20871   3.20871   3.20871   3.20871
 5.0  2.48448   1.85344   1.67286   1.62895     1.95349   1.95349   1.95349   1.95349

julia> org_sol= Array(sol)[important_specs[1],:]   # original solution
17×551 Matrix{Float32}:
 5.0  4.66347   4.35687   4.08074   3.83331   …  2.04756  2.04756  2.04756  2.04756  2.04756
 5.0  4.76789   4.56304   4.37886   4.21144      1.37875  1.37875  1.37875  1.37875  1.37875
 0.0  0.164383  0.255233  0.324004  0.386634     3.27642  3.27642  3.27642  3.27642  3.27642
 0.0  0.336533  0.64313   0.91926   1.16669      2.95244  2.95244  2.95244  2.95244  2.95244
 5.0  4.85251   4.66501   4.45773   4.25667      2.74103  2.74103  2.74103  2.74103  2.74103
 0.0  0.751566  1.23408   1.47399   1.60766   …  1.90095  1.90095  1.90095  1.90095  1.90095
 0.0  1.26558   1.35919   1.43976   1.51766      1.90095  1.90095  1.90095  1.90095  1.90095
 5.0  1.81069   1.79149   1.79129   1.79129      1.79129  1.79129  1.79129  1.79129  1.79129
 5.0  1.81069   1.79149   1.79129   1.79129      1.79129  1.79129  1.79129  1.79129  1.79129
 0.0  3.18931   3.20851   3.20871   3.20871      3.20871  3.20871  3.20871  3.20871  3.20871
 5.0  4.76789   4.56304   4.37886   4.21144   …  1.37875  1.37875  1.37875  1.37875  1.37875
 0.0  0.800619  1.25992   1.46779   1.58404      1.90095  1.90095  1.90095  1.90095  1.90095
 5.0  1.81069   1.79149   1.79129   1.79129      1.79129  1.79129  1.79129  1.79129  1.79129
 5.0  1.81069   1.79149   1.79129   1.79129      1.79129  1.79129  1.79129  1.79129  1.79129
 0.0  3.18931   3.20851   3.20871   3.20871      3.20871  3.20871  3.20871  3.20871  3.20871
 5.0  1.81069   1.79149   1.79129   1.79129   …  1.79129  1.79129  1.79129  1.79129  1.79129
 5.0  1.81069   1.79149   1.79129   1.79129      1.79129  1.79129  1.79129  1.79129  1.79129

julia> optfunc = GalacticOptim.instantiate_function(optf, probs[1].p, adtype, nothing)
(::OptimizationFunction{false, GalacticOptim.AutoZygote, OptimizationFunction{true, GalacticOptim.AutoZygote, var"#23#24", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, GalacticOptim.var"#254#264"{GalacticOptim.var"#253#263"{OptimizationFunction{true, GalacticOptim.AutoZygote, var"#23#24", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Nothing}}, GalacticOptim.var"#257#267"{GalacticOptim.var"#253#263"{OptimizationFunction{true, GalacticOptim.AutoZygote, var"#23#24", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Nothing}}, GalacticOptim.var"#262#272", Nothing, Nothing, Nothing}) (generic function with 1 method)

julia>     optprob = GalacticOptim.OptimizationProblem(optfunc, probs[1].p)
OptimizationProblem. In-place: false
u0: 18-element Vector{Float32}:
 0.3274659
 0.94486237
 1.0
 1.0
 1.0
 1.0
 0.00990099
 0.00990099
 1.0
 1.0
 1.0
 1.0
 1.0
 1.0
 0.051807642
 0.6663984
 1.0
 1.0

julia> result = GalacticOptim.solve(optprob, ADAM(0.05),cb = callback, maxiters = 300)
ERROR: MethodError: Cannot `convert` an object of type Nothing to an object of type Float32
Closest candidates are:
  convert(::Type{T}, ::ChainRulesCore.AbstractZero) where T<:Number at /home/chetan/.julia/packages/ChainRulesCore/7ZiwT/src/tangent_types/abstract_zero.jl:27
  convert(::Type{N}, ::DomainSets.Point{var"#s13"} where var"#s13"<:Number) where N<:Number at /home/chetan/.julia/packages/DomainSets/F4hmL/src/domains/point.jl:14
  convert(::Type{T}, ::Base.TwicePrecision) where T<:Number at twiceprecision.jl:250
  ...
Stacktrace:
  [1] fill!(dest::Vector{Float32}, x::Nothing)
    @ Base ./array.jl:333
  [2] copyto!
    @ ./broadcast.jl:944 [inlined]
  [3] materialize!
    @ ./broadcast.jl:894 [inlined]
  [4] materialize!(dest::Vector{Float32}, bc::Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{0}, Nothing, typeof(identity), Tuple{Base.RefValue{Nothing}}})
    @ Base.Broadcast ./broadcast.jl:891
  [5] (::GalacticOptim.var"#254#264"{GalacticOptim.var"#253#263"{OptimizationFunction{true, GalacticOptim.AutoZygote, var"#23#24", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Nothing}})(::Vector{Float32}, ::Vector{Float32})
    @ GalacticOptim ~/.julia/packages/GalacticOptim/DHxE0/src/function/zygote.jl:8
  [6] macro expansion
    @ ~/.julia/packages/GalacticOptim/DHxE0/src/solve/flux.jl:41 [inlined]
  [7] macro expansion
    @ ~/.julia/packages/GalacticOptim/DHxE0/src/utils.jl:35 [inlined]
  [8] __solve(prob::OptimizationProblem{false, OptimizationFunction{false, GalacticOptim.AutoZygote, OptimizationFunction{true, GalacticOptim.AutoZygote, var"#23#24", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, GalacticOptim.var"#254#264"{GalacticOptim.var"#253#263"{OptimizationFunction{true, GalacticOptim.AutoZygote, var"#23#24", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Nothing}}, GalacticOptim.var"#257#267"{GalacticOptim.var"#253#263"{OptimizationFunction{true, GalacticOptim.AutoZygote, var"#23#24", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Nothing}}, GalacticOptim.var"#262#272", Nothing, Nothing, Nothing}, Vector{Float32}, SciMLBase.NullParameters, Nothing, Nothing, Nothing, Nothing, Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}}, opt::ADAM, data::Base.Iterators.Cycle{Tuple{GalacticOptim.NullData}}; maxiters::Int64, cb::Function, progress::Bool, save_best::Bool, kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ GalacticOptim ~/.julia/packages/GalacticOptim/DHxE0/src/solve/flux.jl:39
  [9] #solve#476
    @ ~/.julia/packages/SciMLBase/x3z0g/src/solve.jl:3 [inlined]
 [10] top-level scope
    @ none:1

ChrisRackauckas · November 11, 2021, 10:28am

Your loss function doesn’t use p.

yewalenikhil65 · November 12, 2021, 3:58am

Ohh, thanks for that, I forgot to put predict_ode line inside loss function.
By the way, does sciml_train allow loss function to have multiple arguments ? say other than just p?

ChrisRackauckas · November 12, 2021, 6:08am

no

vettert · November 12, 2021, 8:26am

The short answer is no, but p can of course contain as many things as you want (tuple, array, …). So, as long as your loss function can make sense of it and it fulfills the required properties, then this is a highly flexible construct.

yewalenikhil65 · November 12, 2021, 8:39am

thanks for the help, i will try writing loss this way!

Topic		Replies	Views
Problem using ModellingToolkit and Galacticoptim to solve a vector/matrix optimization problem General Usage	6	400	June 3, 2022
Sciml_train with BFGS() (post-training with ADAM)causes following error General Usage diffeq , sciml	6	760	November 27, 2021
GalacticOptim trips over boundary conditions in DiffEqOperators Modelling & Simulations diffeq , flux , sciml , ad	2	650	February 11, 2021
Getting issue in optim.optimize function while bayesian estimating parameters of stochastic differential equation New to Julia	1	275	October 12, 2022
Trouble writing OptimizationFunction for automatic forward differentiation during Parameter Estimation of an ODEProblem New to Julia ode , optimization , sciml , autodiff	2	439	January 25, 2023

Using GalacticOptim to optimize, cannot understand the error

Related topics