What is a good tool to track the memory usage of the Julia process over time? Ideally, I would like to find out how much memory is used by Julia at any given time, especially at peak times during, e.g., compilation, but also during the run of an application. My motivation in this case is not to fin…

The allocation profiler may be useful here to visualise the memory footprint of your application during runtime. There’s a good Julia Con talk - Hunting down allocations with Julia 1.8's Allocation Profiler | JuliaCon 2022 I don’t think this will be exactly what you want, and I’m interested to see …

How about logging the high water mark as reported by the ps command? Run it in a loop with sleep of a second or so?

[image] jmair: The allocation profiler may be useful here to visualise the memory footprint of your application during runtime. There’s a good Julia Con talk - Hunting down allocations with Julia 1.8’s Allocation Profiler | JuliaCon 2022 Thanks for the link, it was insightful to watch! Howev…

FYI, I had some very insightful discussions with @vchuravy on Slack, some of which I would like to share here (and to record it for future reference): He suggested a neat trick using a self-resurrecting finalizer to automatically print the maximum RSS (resident set size as reported by Julia’s Sys.…

Based on @vchuravy suggestions above, I gave it a shot and came up with two implementations in Julia. The first one meminfo_julia() only uses built-in functions and thus should be relatively portable, while the second one meminfo_procfs() queries the procfs filesystem directly: using Printf: @prin…

It seems the method of @vchuravy (pure genius!) is going to tell us the high-water mark at each GC collection. Isn’t that superior to something that you have to call yourself?

[image] sloede: function setup_memuse_tracker() tracker = Ref(0) function mem_use(tracker) finalizer(mem_use, tracker) @info "Current memory used" Sys.maxrss() nothing end finalizer(mem_use, tracker) …

Shouldn’t the first line in meminfo_julia() have Base.gc_total_bytes()?

Yes, you are right. In fact, it should read Base.gc_total_bytes(gc_num()) since gc_total_bytes expects and argument of type GC_Num. However, I found that this usually underreported memory usage, thus I left it out after some initial tests. The implementation above was not properly cleaned up, thanks…

How to track total memory usage of Julia process over time

Tooling Profiling

sloede December 6, 2022, 1:04pm 9

Hm, it works for me with Julia v1.8.3 on Linux and macOS. Which OS and Julia version are you using?

Limit / Measure Juliaprogram Memory / RAM consumption

Topic		Replies	Views
Yet another question about Julia's high RAM usage Internals & Design memory	26	2733	July 24, 2026
Limit / Measure Juliaprogram Memory / RAM consumption New to Julia	3	780	July 9, 2023
Multithreaded memory usage General Usage	19	2133	July 14, 2023
How to get current Julia process memory usage? General Usage	10	3445	October 22, 2024
Julia GC, heap fragmentation, out of memory, push!/append! Internals & Design	30	3169	May 20, 2024

How to track total memory usage of Julia process over time

Related topics