Significant allocations with Callbacks with lazy interpolation (Vern methods)

Specific Domains Modelling & Simulations
1

I have a question that probably has a simple answer and is more conceptual, so I haven’t yet written a MWE.

I have a small system with low error tolerance that, ignoring any callbacks, is reasonably efficient in terms of allocations

@benchmark solve(testprob,Vern9(),save_everystep=false,reltol=1e-9,abstol=1e-9)

BenchmarkTools.Trial: 
  memory estimate:  9.11 KiB
  allocs estimate:  22
  --------------
  minimum time:     21.803 ms (0.00% GC)
  median time:      22.289 ms (0.00% GC)
  mean time:        22.467 ms (0.00% GC)
  maximum time:     24.702 ms (0.00% GC)
  --------------
  samples:          223
  evals/sample:     1

I also have a callback that uses terminate!(integrator) when some variable gets too large. With this callback, the time to solve is obviously much shorter, but the allocations are much larger:

BenchmarkTools.Trial: 
  memory estimate:  638.00 KiB
  allocs estimate:  11803
  --------------
  minimum time:     566.178 μs (0.00% GC)
  median time:      588.638 μs (0.00% GC)
  mean time:        645.192 μs (7.58% GC)
  maximum time:     8.254 ms (92.18% GC)
  --------------
  samples:          7743
  evals/sample:     1

Is there a simple way to avoid these allocations? I.e, should I be doing anything special with the setup of the callback functions beyond stating the condition and affect!? They currently read something like:

function condition(out,u,t,integrator)
  out[1] = ... #variable 1 zerocrossing
  out[2] = ... #variable 2 zerocrossing
end

affect!(integrator,idx) = terminate!(integrator)

cb = VectorContinuousCallback(condition,affect!,2)
1 Like
2

That comes from the lazy interpolation of the Verner methods. They shouldn’t be there if you use something like Tsit5 (at least on the latest DiffEqBase). DP8 might also be fine?

Improving the lazy interpolation code could in theory be done for this specific case.

3

It seems like Tsit5 also have this issue. I had the same problem where I used callback for terminating the solver when some values get too large. I used the Tsit5 solver. Without callback I have (43 allocations: 16.23 KiB), and with callback I have (42,065,928 allocations: 3.13 GiB). The callback takes more than 40% of the time. Is there any solution for this lazy interpolation issue?

4

That doesn’t have a lazy interpolation, so it wouldn’t be the same issue. Is your callback code type stable?

5

I didn’t know that callback function could be type unstable, since everything seems to be pre-defined. But here’s the @code_warntype output (seems like type stable to me):

Variables
  #self#::Type{VectorContinuousCallback}
  condition::Core.Compiler.Const(condition, false)
  affect!::Core.Compiler.Const(affect!, false)
  affect_neg!::Core.Compiler.Const(nothing, false)
  len::Int64

Body::VectorContinuousCallback{typeof(condition),typeof(affect!),Nothing,typeof(DiffEqBase.INITIALIZE_DEFAULT),typeof(DiffEqBase.FINALIZE_DEFAULT),Float64,Int64,Rational{Int64},Nothing,Int64}
1 ─ %1 = Core.tuple(true, true)::Core.Compiler.Const((true, true), false)
│   %2 = DiffEqBase.eps()::Core.Compiler.Const(2.220446049250313e-16, false)
│   %3 = (10 * %2)::Core.Compiler.Const(2.220446049250313e-15, false)
│   %4 = (1 // 100)::Rational{Int64}
│   %5 = DiffEqBase.:(var"#VectorContinuousCallback#21")(DiffEqBase.INITIALIZE_DEFAULT, DiffEqBase.FINALIZE_DEFAULT, DiffEqBase.nothing, DiffEqBase.LeftRootFind, %1, 10, 1, %3, 0, %4, #self#, condition, affect!, affect_neg!, len)::VectorContinuousCallback{typeof(condition),typeof(affect!),Nothing,typeof(DiffEqBase.INITIALIZE_DEFAULT),typeof(DiffEqBase.FINALIZE_DEFAULT),Float64,Int64,Rational{Int64},Nothing,Int64}
└──      return %5
6

The callback you use seems to enable additional saving of the solution by default, see Event Handling and Callback Functions · DifferentialEquations.jl. Try setting save_positions=(false, false) in the constructor. Does that help? If not, could you provide a runnable minimal working example?

7

That’s just the callback constructor, so that’s not going to say anything useful. It’s going to be about the affect! function itself. You’ll need to share code. But make this a new thread because it’s unrelated to this.

8

And BTW, to finish this thread, the lazy behavior can be turned off with Vern9(lazy=false), though non-lazy Verner is much less efficient in memory and so using DP8 would still be recommended. Lazy interpolation only applies to Vern7, Vern8, and Vern9 integrators.

9

Thanks. Reposted here: Significant allocations with Callbacks (Tsit5)

10

Thanks! This reduced a tiny bit of allocations, but there is still a whole lot more.