Using GalacticOptim to optimize, cannot understand the error

General Usage
, ,
1

I have a vector of ODE problems that i want to optimize, and here is first odeproblem from the probs array which contains my ODEProblem. I am using GalacticOptim to optimize, but am encountering following error. Some code is also attached. What could be the issue here ? I did not understand the error.! Let me know if some more explanation is need!

function loss(p)
    loss = sum(abs2, soln .- org_sol)
    return loss, soln
end
callback = function (p, l, pred)
    display(l)
    #plt = plot(pred)
    #display(plt)
    return false
end
# use GalacticOptim.jl to solve the problem
adtype = GalacticOptim.AutoZygote()
optf = GalacticOptim.OptimizationFunction((x, p) -> loss(x), adtype)

julia> probs[1].p   # parameters to be optimized
18-element Vector{Float32}:
 0.3274659
 0.94486237
 1.0
 1.0
 1.0
 1.0
 0.00990099
 0.00990099
 1.0
 1.0
 1.0
 1.0
 1.0
 1.0
 0.051807642
 0.6663984
 1.0
 1.0

julia> soln = Array(solve(probs[1], p = probs[1].p, saveat = 1.0))  # approximated solution
17×551 Matrix{Float32}:
 5.0  4.18527   4.08949   4.12522   4.18547  …  5.63654   5.63654   5.63654   5.63654
 5.0  4.76808   4.56416   4.38125   4.21494     1.39767   1.39767   1.39767   1.39767
 0.0  0.124666  0.142363  0.138717  0.13336     0.151869  0.151869  0.151869  0.15187
 0.0  1.15974   1.42228   1.48669   1.51855     1.95349   1.95349   1.95349   1.95349
 0.0  1.41916   1.53627   1.58009   1.60998     1.95349   1.95349   1.95349   1.95349
 5.0  1.81066   1.79148   1.79129   1.79129  …  1.79129   1.79129   1.79129   1.79129
 5.0  1.81066   1.79148   1.79129   1.79129     1.79129   1.79129   1.79129   1.79129
 0.0  3.18934   3.20852   3.20871   3.20871     3.20871   3.20871   3.20871   3.20871
 5.0  4.76808   4.56416   4.38125   4.21494     1.39767   1.39767   1.39767   1.39767
 0.0  0.8586    1.39199   1.61517   1.70875     1.95349   1.95349   1.95349   1.95349
 5.0  1.81066   1.79148   1.79129   1.79129  …  1.79129   1.79129   1.79129   1.79129
 5.0  1.81066   1.79148   1.79129   1.79129     1.79129   1.79129   1.79129   1.79129
 0.0  3.18934   3.20852   3.20871   3.20871     3.20871   3.20871   3.20871   3.20871
 5.0  1.81066   1.79148   1.79129   1.79129     1.79129   1.79129   1.79129   1.79129
 5.0  1.81066   1.79148   1.79129   1.79129     1.79129   1.79129   1.79129   1.79129
 0.0  3.18934   3.20852   3.20871   3.20871  …  3.20871   3.20871   3.20871   3.20871
 5.0  2.48448   1.85344   1.67286   1.62895     1.95349   1.95349   1.95349   1.95349

julia> org_sol= Array(sol)[important_specs[1],:]   # original solution
17×551 Matrix{Float32}:
 5.0  4.66347   4.35687   4.08074   3.83331   …  2.04756  2.04756  2.04756  2.04756  2.04756
 5.0  4.76789   4.56304   4.37886   4.21144      1.37875  1.37875  1.37875  1.37875  1.37875
 0.0  0.164383  0.255233  0.324004  0.386634     3.27642  3.27642  3.27642  3.27642  3.27642
 0.0  0.336533  0.64313   0.91926   1.16669      2.95244  2.95244  2.95244  2.95244  2.95244
 5.0  4.85251   4.66501   4.45773   4.25667      2.74103  2.74103  2.74103  2.74103  2.74103
 0.0  0.751566  1.23408   1.47399   1.60766   …  1.90095  1.90095  1.90095  1.90095  1.90095
 0.0  1.26558   1.35919   1.43976   1.51766      1.90095  1.90095  1.90095  1.90095  1.90095
 5.0  1.81069   1.79149   1.79129   1.79129      1.79129  1.79129  1.79129  1.79129  1.79129
 5.0  1.81069   1.79149   1.79129   1.79129      1.79129  1.79129  1.79129  1.79129  1.79129
 0.0  3.18931   3.20851   3.20871   3.20871      3.20871  3.20871  3.20871  3.20871  3.20871
 5.0  4.76789   4.56304   4.37886   4.21144   …  1.37875  1.37875  1.37875  1.37875  1.37875
 0.0  0.800619  1.25992   1.46779   1.58404      1.90095  1.90095  1.90095  1.90095  1.90095
 5.0  1.81069   1.79149   1.79129   1.79129      1.79129  1.79129  1.79129  1.79129  1.79129
 5.0  1.81069   1.79149   1.79129   1.79129      1.79129  1.79129  1.79129  1.79129  1.79129
 0.0  3.18931   3.20851   3.20871   3.20871      3.20871  3.20871  3.20871  3.20871  3.20871
 5.0  1.81069   1.79149   1.79129   1.79129   …  1.79129  1.79129  1.79129  1.79129  1.79129
 5.0  1.81069   1.79149   1.79129   1.79129      1.79129  1.79129  1.79129  1.79129  1.79129

julia> optfunc = GalacticOptim.instantiate_function(optf, probs[1].p, adtype, nothing)
(::OptimizationFunction{false, GalacticOptim.AutoZygote, OptimizationFunction{true, GalacticOptim.AutoZygote, var"#23#24", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, GalacticOptim.var"#254#264"{GalacticOptim.var"#253#263"{OptimizationFunction{true, GalacticOptim.AutoZygote, var"#23#24", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Nothing}}, GalacticOptim.var"#257#267"{GalacticOptim.var"#253#263"{OptimizationFunction{true, GalacticOptim.AutoZygote, var"#23#24", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Nothing}}, GalacticOptim.var"#262#272", Nothing, Nothing, Nothing}) (generic function with 1 method)

julia>     optprob = GalacticOptim.OptimizationProblem(optfunc, probs[1].p)
OptimizationProblem. In-place: false
u0: 18-element Vector{Float32}:
 0.3274659
 0.94486237
 1.0
 1.0
 1.0
 1.0
 0.00990099
 0.00990099
 1.0
 1.0
 1.0
 1.0
 1.0
 1.0
 0.051807642
 0.6663984
 1.0
 1.0

julia> result = GalacticOptim.solve(optprob, ADAM(0.05),cb = callback, maxiters = 300)
ERROR: MethodError: Cannot `convert` an object of type Nothing to an object of type Float32
Closest candidates are:
  convert(::Type{T}, ::ChainRulesCore.AbstractZero) where T<:Number at /home/chetan/.julia/packages/ChainRulesCore/7ZiwT/src/tangent_types/abstract_zero.jl:27
  convert(::Type{N}, ::DomainSets.Point{var"#s13"} where var"#s13"<:Number) where N<:Number at /home/chetan/.julia/packages/DomainSets/F4hmL/src/domains/point.jl:14
  convert(::Type{T}, ::Base.TwicePrecision) where T<:Number at twiceprecision.jl:250
  ...
Stacktrace:
  [1] fill!(dest::Vector{Float32}, x::Nothing)
    @ Base ./array.jl:333
  [2] copyto!
    @ ./broadcast.jl:944 [inlined]
  [3] materialize!
    @ ./broadcast.jl:894 [inlined]
  [4] materialize!(dest::Vector{Float32}, bc::Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{0}, Nothing, typeof(identity), Tuple{Base.RefValue{Nothing}}})
    @ Base.Broadcast ./broadcast.jl:891
  [5] (::GalacticOptim.var"#254#264"{GalacticOptim.var"#253#263"{OptimizationFunction{true, GalacticOptim.AutoZygote, var"#23#24", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Nothing}})(::Vector{Float32}, ::Vector{Float32})
    @ GalacticOptim ~/.julia/packages/GalacticOptim/DHxE0/src/function/zygote.jl:8
  [6] macro expansion
    @ ~/.julia/packages/GalacticOptim/DHxE0/src/solve/flux.jl:41 [inlined]
  [7] macro expansion
    @ ~/.julia/packages/GalacticOptim/DHxE0/src/utils.jl:35 [inlined]
  [8] __solve(prob::OptimizationProblem{false, OptimizationFunction{false, GalacticOptim.AutoZygote, OptimizationFunction{true, GalacticOptim.AutoZygote, var"#23#24", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, GalacticOptim.var"#254#264"{GalacticOptim.var"#253#263"{OptimizationFunction{true, GalacticOptim.AutoZygote, var"#23#24", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Nothing}}, GalacticOptim.var"#257#267"{GalacticOptim.var"#253#263"{OptimizationFunction{true, GalacticOptim.AutoZygote, var"#23#24", Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Nothing}}, GalacticOptim.var"#262#272", Nothing, Nothing, Nothing}, Vector{Float32}, SciMLBase.NullParameters, Nothing, Nothing, Nothing, Nothing, Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}}, opt::ADAM, data::Base.Iterators.Cycle{Tuple{GalacticOptim.NullData}}; maxiters::Int64, cb::Function, progress::Bool, save_best::Bool, kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ GalacticOptim ~/.julia/packages/GalacticOptim/DHxE0/src/solve/flux.jl:39
  [9] #solve#476
    @ ~/.julia/packages/SciMLBase/x3z0g/src/solve.jl:3 [inlined]
 [10] top-level scope
    @ none:1
2

Your loss function doesn’t use p.

1 Like
3

Ohh, thanks for that, I forgot to put predict_ode line inside loss function.
By the way, does sciml_train allow loss function to have multiple arguments ? say other than just p?

4

no

5

The short answer is no, but p can of course contain as many things as you want (tuple, array, …). So, as long as your loss function can make sense of it and it fulfills the required properties, then this is a highly flexible construct.

1 Like
6

thanks for the help, i will try writing loss this way!