How to track total memory usage of Julia process over time

sloede
Sat, 03 Dec 2022 06:06:13 +0000
Profiling

What is a good tool to track the memory usage of the Julia process over time? Ideally, I would like to find out how much memory is used by Julia at any given time, especially at peak times during, e.g., compilation, but also during the run of an application.

My motivation in this case is not to find out which part of the code allocates how much for performance reasons, but what the overall memory footprint of the Julia process is. The rationale is that when doing parallel runs with MPI.jl, we sometimes run into out-of-memory issues.

Our particular example: On a single node with 256 GiB of main memory we get an OOM error when running a simulation with Trixi.jl on 128 MPI ranks, even though the total application memory use is (conservatively) estimated to be < 50 GiB. Thus it seems that the memory overhead of the Julia process itself is more than 1 GiB (maybe due to memory required during the initial compilation pass?), but we would like to find out for sure.

18 posts - 6 participants