How to track total memory usage of Julia process over time

Tooling Profiling
5

FYI, I had some very insightful discussions with @vchuravy on Slack, some of which I would like to share here (and to record it for future reference):

  1. He suggested a neat trick using a self-resurrecting finalizer to automatically print the maximum RSS (resident set size as reported by Julia’s Sys.maxrss(), which internally uses the C function getrusage):
julia> function setup_memuse_tracker()
          tracker = Ref(0)
          function mem_use(tracker)
            finalizer(mem_use, tracker)
            @info "Current memory used" Sys.maxrss()
            nothing
          end
          
          finalizer(mem_use, tracker)
          nothing
       end
setup_memuse_tracker (generic function with 1 method)

julia> setup_memuse_tracker()

julia> GC.gc()
┌ Info: Current memory used
└   Sys.maxrss() = 0x000000001280b000
  1. He suggested looking at the procfs file system, e.g., /proc/<pid>/statm. This is similar to the suggestion of using ps by @PetrKryslUCSD, but could forgo the need for an external program.
  2. He further suggested to look at the output of Base.gc_num(). Alternatively, at the values of Base.gc_live_bytes() and/or Base.jit_total_bytes().
  3. Finally he suggested to take a look at the tool Arm Forge, which combines the Arm DDT/Arm MAP profiling tools. However, one has to pay for those.
4 Likes