Hello,
I am trying to optimize a simple linear system towards some known parameters and switched from the ADAMs optimizer to (L)BFGS. I didn’t change the problem/ode but for some reason the (L)BFGS optimizer runs in severe stabilitiy problems. These problems always occur right of the start, no matter which initial point I choose.
I found a blog post where the same porblem was described and solved
→ Handling Instability When Solving ODE Problems
and tried some of the proposed methods, but neither turning autodiff of or using “CVODE_BDF” improved the situation.
Here is some example code:
function oscillator_linear(du, u, p, t)
x1, y1, x2, y2 = u
α, δ = p
β = 0
γ = 0
du[1] = y1
du[2] = broadband(t) - α*y1 - β*x1*x1*y1 - x1 - γ*x1*x1*x1 + δ*(x2-x1)
du[3] = y2
du[4] = - α*y2 - β*x2*x2*y2 - x2 - γ*x2*x2*x2 - δ*(x2-x1)
end
p = [1.0, 1.0]
u0 = [0.0, 0.0, 0.0, 0.0]
# Define the ODE problem
tspan = (0.0, time_training[end])
prob = ODEProblem(oscillator_linear, u0, tspan, p)
# Define all required functions
function predict_rd(par)
sol = solve(prob,CVODE_BDF(),p=par,saveat=t_step)
return sol[3,:]
end
# Loss function
function loss_rd(par)
sol = predict_rd(par)
loss = sum(abs2,sol-Q_training)
# Edits for debugging
println("Loss: $loss")
par1 = par[1]
par2 = par[2]
display("Par: $par1 - $par2")
return loss
end
# Begin the linear optimization
println("Starting Linear Optimization")
opt = optimize(loss_rd, p, BFGS())
The console output looks like:
Loss: 1715.2924527208536
“Par: 1.0000060554544523 - 1.0”
Loss: 1714.3889825218346
“Par: 0.9999939445455476 - 1.0”
Loss: 1714.0457216321786
“Par: 1.0 - 1.0000060554544523”
Loss: 1713.7411094002068
“Par: 1.0 - 0.9999939445455476”
Loss: 1714.964781270175
“Par: 1.0 - 1.0”
[CVODES WARNING] CVode
Internal t = 0.00947827 and h = 2.36212e-020 are such that t + h
= t on the next step. The solver will continue anyway.
[CVODES WARNING] CVode
Internal t = 0.00947827 and h = 5.9053e-021 are such that t + h = t on the next step. The solver will continue anyway.
[CVODES WARNING] CVode
Internal t = 0.00947827 and h = 5.9053e-021 are such that t + h = t on the next step. The solver will continue anyway.
[CVODES WARNING] CVode
Internal t = 0.00947827 and h = 3.69081e-021 are such that t + h
= t on the next step. The solver will continue anyway.
[CVODES WARNING] CVode
Internal t = 0.00947827 and h = 3.69081e-021 are such that t + h
= t on the next step. The solver will continue anyway.
[CVODES ERROR] CVode
At t = 0.00947827 and h = 1.40793e-025, the corrector convergence test failed repeatedly or with |h| = hmin.
ERROR: LoadError: DimensionMismatch(“dimensions must match: a has dims (Base.OneTo(2),), b has dims (Base.OneTo(40000),), mismatch at
1”)