Stability Issues in Optim

Hello,

I am trying to optimize a simple linear system towards some known parameters and switched from the ADAMs optimizer to (L)BFGS. I didn’t change the problem/ode but for some reason the (L)BFGS optimizer runs in severe stabilitiy problems. These problems always occur right of the start, no matter which initial point I choose.

I found a blog post where the same porblem was described and solved
Handling Instability When Solving ODE Problems
and tried some of the proposed methods, but neither turning autodiff of or using “CVODE_BDF” improved the situation.

Here is some example code:

function oscillator_linear(du, u, p, t)
  x1, y1, x2, y2 = u
  α, δ = p
  β = 0
  γ = 0
  du[1] = y1
  du[2] = broadband(t) - α*y1 - β*x1*x1*y1 - x1 - γ*x1*x1*x1 + δ*(x2-x1)
  du[3] = y2
  du[4] = - α*y2 - β*x2*x2*y2 - x2 - γ*x2*x2*x2 - δ*(x2-x1)
end

p = [1.0, 1.0]
u0 = [0.0, 0.0, 0.0, 0.0]
# Define the ODE problem
tspan = (0.0, time_training[end])
prob = ODEProblem(oscillator_linear, u0, tspan, p)

# Define all required functions
function predict_rd(par)
    sol = solve(prob,CVODE_BDF(),p=par,saveat=t_step)
    return sol[3,:]
end

# Loss function
function loss_rd(par)
  sol = predict_rd(par)
  loss = sum(abs2,sol-Q_training)
  # Edits for debugging 
  println("Loss: $loss")
  par1 = par[1]
  par2 = par[2]
  display("Par: $par1 - $par2")
  return loss
end

# Begin the linear optimization
println("Starting Linear Optimization")
opt = optimize(loss_rd, p, BFGS())

The console output looks like:
Loss: 1715.2924527208536
“Par: 1.0000060554544523 - 1.0”
Loss: 1714.3889825218346
“Par: 0.9999939445455476 - 1.0”
Loss: 1714.0457216321786
“Par: 1.0 - 1.0000060554544523”
Loss: 1713.7411094002068
“Par: 1.0 - 0.9999939445455476”
Loss: 1714.964781270175
“Par: 1.0 - 1.0”

[CVODES WARNING] CVode
Internal t = 0.00947827 and h = 2.36212e-020 are such that t + h
= t on the next step. The solver will continue anyway.

[CVODES WARNING] CVode
Internal t = 0.00947827 and h = 5.9053e-021 are such that t + h = t on the next step. The solver will continue anyway.

[CVODES WARNING] CVode
Internal t = 0.00947827 and h = 5.9053e-021 are such that t + h = t on the next step. The solver will continue anyway.

[CVODES WARNING] CVode
Internal t = 0.00947827 and h = 3.69081e-021 are such that t + h
= t on the next step. The solver will continue anyway.

[CVODES WARNING] CVode
Internal t = 0.00947827 and h = 3.69081e-021 are such that t + h
= t on the next step. The solver will continue anyway.

[CVODES ERROR] CVode
At t = 0.00947827 and h = 1.40793e-025, the corrector convergence test failed repeatedly or with |h| = hmin.

ERROR: LoadError: DimensionMismatch(“dimensions must match: a has dims (Base.OneTo(2),), b has dims (Base.OneTo(40000),), mismatch at
1”)