Estimate memory requirements of ODE solver

Salmon
Wed, 04 Oct 2023 10:25:54 +0000
General Usage

Hi everyone,
I am solving a very large (~10^8) system of ODE’s. I would like to use the VCABM solver from ordinaryDiffEq since that seems to be fastest, but I have memory issues that I would like to understand better.

I am aware that the ODE solvers will allocate their cache dependent on the order.
It looks like VCABM needs quite a few caches about 60, looking at the implementation in OrdinaryDiffEq:
alg_cache (click for more details)
I confirmed this by printing sizeofcache = Base.format_bytes(Base.summarysize(cache)) (gives 64 times the size of my state array).

This is quite a lot, but should still fit in my memory (for 100 caches the size of my input array, I should be only at 56 GB memory).
Nonetheless, I find my calculation to consume 2-3 times that amount (~133 GB).

Can anyone give me a pointer if this is expected? How do I find out why so much memory is allocated? A few more points:

I use save_everystep = false. More generally, the memory allocation does not increase after the simulation is started, so it is no memory leak

I realize that arrays may allocate more memory than they need, but I believe this should not be an issue here, since I allocate my state with zeros(dims…)

8 posts - 2 participants