Bounded sciml_train problem error, ArgumentError: Initial x is not in

Hi,

I have been getting this initialization error when I try to do iterative box optimization with DiffEqFlux.sciml_train. I am attempting to optimize sets of parameters for which the loss function slightly varies for different runs (but constant bounds). I try to pass the optimized parameters from one run to the next but hit this “initialization out of bounds” error despite clamping the input and output of sciml_train… What am I doing wrong?

Below, is a one dimensional MWE which recreates this error. Note, that the loss fn doesn’t vary in this example, but I wanted to keep it in the opt_fun to mimic full script.

using DiffEqFlux

n_d, n_opt = 1, 3

function opt_fun(p)

    loss = p -> p.^2
    lower_bounds, upper_bounds = 0.5*ones(n_d), 2.0*ones(n_d)
    its = 2

    p = clamp.(p, lower_bounds, upper_bounds)
    result = DiffEqFlux.sciml_train(loss,
                p;
                lower_bounds, upper_bounds,
                maxiters = its
                )
    p_min = clamp.(result.minimizer, lower_bounds, upper_bounds)

    return p_min
end

p_mat = ones(n_d, n_opt)

for i=1:n_opt-1
    global p_mat
    println(p_mat[:, i])
    p_mat[:, i+1] = opt_fun(p_mat[:, i])
end

outputs:

[1.0]
[0.5000000006608806]
ERROR: ArgumentError: Initial x[(1,)]=0.48000577652855314 is outside of [0.5, 2.0]
Stacktrace:
 [1] optimize(df::OnceDifferentiable{Float64, Vector{Float64}, Vector{Float64}}, l::Vector{Float64}, u::Vector{Float64}, initial_x::Vector{Float64}, F::Fminbox{BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Float64, Flat}, Float64, Optim.var"#47#49"}, options::Optim.Options{Float64, GalacticOptim.var"#_cb#150"{GalacticOptim.var"#149#155", Fminbox{BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Float64, Flat}, Float64, Optim.var"#47#49"}, Base.Iterators.Cycle{Tuple{GalacticOptim.NullData}}}})
   @ Optim ~/.julia/packages/Optim/rES57/src/multivariate/solvers/constrained/fminbox.jl:310
 [2] ___solve(prob::SciMLBase.OptimizationProblem{false, SciMLBase.OptimizationFunction{false, GalacticOptim.AutoZygote, SciMLBase.OptimizationFunction{true, GalacticOptim.AutoZygote, DiffEqFlux.var"#82#87"{var"#7#8"}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, GalacticOptim.var"#254#264"{GalacticOptim.var"#253#263"{SciMLBase.OptimizationFunction{true, GalacticOptim.AutoZygote, DiffEqFlux.var"#82#87"{var"#7#8"}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Nothing}}, GalacticOptim.var"#257#267"{GalacticOptim.var"#253#263"{SciMLBase.OptimizationFunction{true, GalacticOptim.AutoZygote, DiffEqFlux.var"#82#87"{var"#7#8"}, Nothing, Nothing, Nothing, Nothing, Nothing, Nothing}, Nothing}}, GalacticOptim.var"#262#272", Nothing, Nothing, Nothing}, Vector{Float64}, SciMLBase.NullParameters, Vector{Float64}, Nothing, Nothing, Nothing, Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}}, opt::Fminbox{BFGS{LineSearches.InitialStatic{Float64}, LineSearches.HagerZhang{Float64, Base.RefValue{Bool}}, Nothing, Float64, Flat}, Float64, Optim.var"#47#49"}, data::Base.Iterators.Cycle{Tuple{GalacticOptim.NullData}}; cb::Function, maxiters::Int64, maxtime::Nothing, abstol::Nothing, reltol::Nothing, progress::Bool, kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
   @ GalacticOptim ~/.julia/packages/GalacticOptim/DHxE0/src/solve/optim.jl:196
 [3] #__solve#127
   @ ~/.julia/packages/GalacticOptim/DHxE0/src/solve/optim.jl:49 [inlined]
 [4] #solve#476
   @ ~/.julia/packages/SciMLBase/x3z0g/src/solve.jl:3 [inlined]
 [5] sciml_train(::var"#7#8", ::Vector{Float64}, ::Nothing, ::Nothing; lower_bounds::Vector{Float64}, upper_bounds::Vector{Float64}, maxiters::Int64, kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
   @ DiffEqFlux ~/.julia/packages/DiffEqFlux/jpIWG/src/train.jl:109
 [6] opt_fun(p::Vector{Float64})
   @ Main ~/Documents/Julia/arkin/optim_bdserror.jl:13
 [7] top-level scope
   @ ./none:4

Julia Version 1.6.3
DiffEqFlux v1.44.0

Fixed in the latest version.

1 Like