Memory leak in Julia 1.11.x

[Miroboru]

I have a simulation written in Julia that I have been developing and running for a few years. It is a rather complex simulation with runtimes measured in days when using multicore computers, and it relies mainly on Flux, OptimizationOptimJL in addition to LinearAlgebra and StaticArrays.

After upgrading to 1.11.x I noticed that the memory use of my simulations more than doubled (in the case where I used Distributed, every process would double in size eventually, but this also happens when running a single process). The increase in memory use is gradual from the start to end of the simulation, with no sudden jumps from what I can see. The outputs of the simulations when using release and lts are otherwise identical.

I have tried to dump the sizes of any arrays or other structures I use as the simulations run, but again, there is no difference between release and lts.

This leads me to think there is a memory leak somewhere in the release version of Julia. As of now I am running release=1.11.3 and lts=1.10.7.

How should I investigate this issue any further? Are there any flags or hidden options that can make Julia check for missing deallocations? There is really no way to do a minimal working example here as the simulation code base is quite large, but I would be open to showing off some of the core loops as they are of a managable size.

[barucden]

There was a similar thread recently:

Memory leak when migrating from Julia 1.10.7 to 1.11.2

Hi, I’m encountering a memory issue in Julia v1.11.2 with a Monte Carlo simulation. The code spans multiple files which makes it hard for me to post it here. I can say that the code doesn’t allocate new memory after the Monte Carlo simulation starts (I have checked this on my PC by monitoring the memory usage). On my PC, memory usage is stable, but on a cluster, it grows significantly (after the simulation begins), leading to OOM errors. This didn’t happen in Julia v1.10.7, where I ran the same…

Maybe it’s related to your issue?

[Miroboru]

Thank you. I will do what this thread suggested and try the nightly build.

[lmiq]

I think this is supposed to be fixed now in 1.11.3 (and 1.10.8). If not, that’s a new issue.

[barucden]

Yes, but the thread mentions a PR (gc: improve mallocarrays locality by vtjnash · Pull Request #56801 · JuliaLang/julia · GitHub) which has more comments.

[lmiq]

Yes, that I understood that was merged and backported to both minor versions.

[Miroboru]

Just checked, and nightly fixes the issue, while 1.11.3 doesn’t. So whatever the problem is the fix hasn’t been backported yet it seems.