DifferentialEquations.jl: Malloc errors in ODEProblems

General Usage
,
1

Hi there,

I’m having memory related problem when running the solver for ODEProblems, with the DifferentialEquations.jl package.

It’s very inconsistent when the problem will arise (I’m scanning across a large number of different parameters that give different RHS to the ODE), but when it does, a typical error message (from a linux cluster):

terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc

[2640520] signal (6.-6): Aborted
in expression starting at none:0
srun: error:  XXX: task 0: Aborted (core dumped)

The fact Julia didn’t throw the error made me question if it’s a problem with the cluster, but I’ve also encountered this on my local MacOS.

A minimal example:

function solver(parameter)
    
    ρ0 = genstate(parameter)
    p = geninput(parameter)

    tspan = (0, 100)
    prob = ODEProblem(lindbladian!, ρ0, tspan, p)
    method = DP8()  # supposedly stable memory wise
    sol = solve(prob, method, reltol = 1e-6, abstol = 1e-6)
end 

Threads.@nthreads for parameter in parameters
    solver(parameter)
end

The function (in its original form)

function lindbladian!(du, u, p, t)

    h = p[1]
    ops = p[2]
    γs = p[3]

    du .= -1im *  commutator(h, u) 

    for (i, op) in enumerate(ops)
        du .+= γs[i] * ( op * u * op' - 1/2 * anticommutator( op' * op, u))
    end 
    nothing
end

I think the details for this function are not too important, but essentially all it does is doing a series of matrix multiplications in place (u is dense, but other matrices h, op are sparse, \gamma is a number) , all with the same size:

Throughout all the examples, u, h, op has dimensions 256 \times 256, which is certainly not small but not to the point of exhausting GBs of RAM (I think?)

The output are not particularly large ( sizeof(sol.u) is well below 1GB), and I’ve played around with different solvers, some definitely performed better than others, but none have been completely free of this problem.

Sorry I currently don’t have much further information to work with, since this problem itself appears somewhat randomly

2

Did you get your Julia binary through juliaup? This seems very odd and would likely be something due to an incorrectly built compiler.

I can’t run this example: it’s missing function definitions.

Are there any c-calls?

3

I think this might be a badly handled out of memory error

4

This is probably unrelated, but I decided to try this on another version of Julia using juliaup (I was on 1.9.4),

I tried installing 1.10.9 and 1.11.4, and as I was installing DifferentialEquations, I got the following error:

`DiffEqNoiseProcess=77a26b50-5914-5dd7-bc55-306e6241c503` depends on 
`ReverseDiff=37e2e3b7-166d-5795-8a7a-e32c996b4267`, but no such entry exists
 in the manifest.

This was a fresh installations, and I’ve tried resetting the environment, etc.

Are there certain versions of julia that I need to use? Sorry if this is a dumb question but I didn’t find such information in the docs

5

That is really odd because DiffEqNoiseProcess does not depend on ReverseDiff.

Are you trying to instantiate from an old manifest?