How to check where memory is used in REPL?

General Usage
1

My package suffers from large memory usage, and I wanna know why by testing in REPL.
I track the total message in two ways:

  1. examing result of top
  2. use the following code in REPL:
    my_command = pipeline(`cat /proc/$(getpid())/statm`, `awk '{print $2}'`)
    run(my_command)
    both ways give me similar results of about 3G.
    Then I wanna know how these memory are used:
julia> varinfo()
  name                    size summary                           
  –––––––––––––––– ––––––––––– ––––––––––––––––––––––––––––––––––
  Base                         Module                            
  Core                         Module                            
  InteractiveUtils 261.479 KiB Module                            
  Main                         Module                            
  ans                1.407 KiB Markdown.MD                       
  c                  1.263 GiB ***(something produced by my package)
  err                  8 bytes Int64                             
  my_command         266 bytes Base.OrCmds

which only covers less than half of the total memory used. How can I find where other memory are used?

I also tried:

julia> varinfo(Revise)
  name               size summary                                   
  ––––––––––––– ––––––––– ––––––––––––––––––––––––––––––––––––––––––
  MethodSummary 216 bytes DataType                                  
  Revise        2.726 MiB Module                                    
  entr            0 bytes entr (generic function with 2 methods)    
  includet        0 bytes includet (generic function with 2 methods)
  revise          0 bytes revise (generic function with 3 methods)

and

julia> varinfo(MyPackage)
  name          size summary
  ––––––– –––––––––– –––––––
  MyPackage 15.182 MiB Module

Which seems to be little.

Strangely, after I go for lunch and return after 30min, the memory usage of the process is now 3.6G. No idea what happened.

It seems to be related be compiled methods, which uses far more memory than I’ve expect. But I don’t know how to get more details.

What I’ve done:

use top to track the memory usage of julia (1.7.1).
after starting a new process: ~200M
after using MyPackage: ~350M
after executing some precompile commands (~250 precompile commands, many third parties library are used): ~780M
after GC: ~720M
some output at this time:

julia> varinfo()
  name                    size summary
  –––––––––––––––– ––––––––––– –––––––
  Base                         Module 
  Core                         Module 
  InteractiveUtils 253.909 KiB Module 
  Main                         Module 
  ans                   1 byte Bool   

julia> Base.gc_live_bytes()
383446566

For solving a real problem, many more methods will be compiled, so in the original problem, maybe more memory are used by compiled codes, but I’ve not find a way to figure out a precise number.

And, after all, it is strange that compiled code should use so many memory. Considering my package is expected to be used with mpi on a cluster with less than 4GB memory per core, this is too high a cost.

2

It could be that the memory is not taken by any variables but was used by variables that went out of scope. The garbage collector is supposed to recover them eventually but you can force it to act by doing:

GC.gc()

In the REPL and then checking again your memory usage.

3

The data reported above is after I run GC.gc(). It only frees about 80M memory.

4

You can try to look at Base.summarysize which tries to give the total size of all objects reachable from a given object. Try running it on all the objects inside varinfo():

julia> x = rand(Float32, 1000, 1000);

julia> s = Dict(i => join(rand('A':'z', 200)) for i in 1:100);

julia> for name in names(Main)
           thing = getproperty(Main, name)
           thing isa Module && continue
           println(String(name), ' ', Base.summarysize(thing))
       end
ans 25336
s 25336
x 4000040

There are probably some edge cases I’ve forgotten about. For example, in my case above, ans and s are the same object, but appears twice in the output.

5

For the main object mentioned above, Base.summarysize also returns about 1.3G.