Stability Issues in Optim

Hello,

I am trying to optimize a simple linear system towards some known parameters and switched from the ADAMs optimizer to (L)BFGS. I didn’t change the problem/ode but for some reason the (L)BFGS optimizer runs in severe stabilitiy problems. These problems always occur right of the start, no matter which initial point I choose.

I found a blog post where the same porblem was described and solved
–> Handling Instability When Solving ODE Problems
and tried some of the proposed methods, but neither turning autodiff of or using “CVODE_BDF” improved the situation.

Here is some example code:

function oscillator_linear(du, u, p, t)
  x1, y1, x2, y2 = u
  α, δ = p
  β = 0
  γ = 0
  du[1] = y1
  du[2] = broadband(t) - α*y1 - β*x1*x1*y1 - x1 - γ*x1*x1*x1 + δ*(x2-x1)
  du[3] = y2
  du[4] = - α*y2 - β*x2*x2*y2 - x2 - γ*x2*x2*x2 - δ*(x2-x1)
end

p = [1.0, 1.0]
u0 = [0.0, 0.0, 0.0, 0.0]
# Define the ODE problem
tspan = (0.0, time_training[end])
prob = ODEProblem(oscillator_linear, u0, tspan, p)

# Define all required functions
function predict_rd(par)
    sol = solve(prob,CVODE_BDF(),p=par,saveat=t_step)
    return sol[3,:]
end

# Loss function
function loss_rd(par)
  sol = predict_rd(par)
  loss = sum(abs2,sol-Q_training)
  # Edits for debugging 
  println("Loss: $loss")
  par1 = par[1]
  par2 = par[2]
  display("Par: $par1 - $par2")
  return loss
end

# Begin the linear optimization
println("Starting Linear Optimization")
opt = optimize(loss_rd, p, BFGS())

The console output looks like:
Loss: 1715.2924527208536
“Par: 1.0000060554544523 - 1.0”
Loss: 1714.3889825218346
“Par: 0.9999939445455476 - 1.0”
Loss: 1714.0457216321786
“Par: 1.0 - 1.0000060554544523”
Loss: 1713.7411094002068
“Par: 1.0 - 0.9999939445455476”
Loss: 1714.964781270175
“Par: 1.0 - 1.0”

[CVODES WARNING] CVode
Internal t = 0.00947827 and h = 2.36212e-020 are such that t + h
= t on the next step. The solver will continue anyway.

[CVODES WARNING] CVode
Internal t = 0.00947827 and h = 5.9053e-021 are such that t + h = t on the next step. The solver will continue anyway.

[CVODES WARNING] CVode
Internal t = 0.00947827 and h = 5.9053e-021 are such that t + h = t on the next step. The solver will continue anyway.

[CVODES WARNING] CVode
Internal t = 0.00947827 and h = 3.69081e-021 are such that t + h
= t on the next step. The solver will continue anyway.

[CVODES WARNING] CVode
Internal t = 0.00947827 and h = 3.69081e-021 are such that t + h
= t on the next step. The solver will continue anyway.

[CVODES ERROR] CVode
At t = 0.00947827 and h = 1.40793e-025, the corrector convergence test failed repeatedly or with |h| = hmin.

ERROR: LoadError: DimensionMismatch(“dimensions must match: a has dims (Base.OneTo(2),), b has dims (Base.OneTo(40000),), mismatch at
1”)