# How to check where memory is used in REPL?

**URL:** https://discourse.julialang.org/t/how-to-check-where-memory-is-used-in-repl/94895
**Category:** General Usage
**Created:** [February 20, 2023, 3:37am UTC](https://discourse.julialang.org/t/how-to-check-where-memory-is-used-in-repl/94895 "2023-02-20T03:37:04Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![jsjie](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jsjie/32/22477_2.png) [@jsjie](https://discourse.julialang.org/u/jsjie)
#### Post date: [February 20, 2023, 3:37am UTC](https://discourse.julialang.org/t/how-to-check-where-memory-is-used-in-repl/94895/1 "2023-02-20T03:37:04Z")

</div>

My package suffers from large memory usage, and I wanna know why by testing in REPL.  
I track the total message in two ways:

1. examing result of `top`
2. use the following code in REPL:  
`my_command = pipeline(`cat /proc/$(getpid())/statm`, `awk '{print $2}'`)`  
` run(my_command)`  
both ways give me similar results of about 3G.  
Then I wanna know how these memory are used:

```julia
julia> varinfo()
  name size summary                           
  –––––––––––––––– ––––––––––– ––––––––––––––––––––––––––––––––––
  Base Module                            
  Core Module                            
  InteractiveUtils 261.479 KiB Module                            
  Main Module                            
  ans 1.407 KiB Markdown.MD                       
  c 1.263 GiB ***(something produced by my package)
  err 8 bytes Int64                             
  my_command 266 bytes Base.OrCmds 

```

which only covers less than half of the total memory used. How can I find where other memory are used?

I also tried:

```julia
julia> varinfo(Revise)
  name size summary                                   
  ––––––––––––– ––––––––– ––––––––––––––––––––––––––––––––––––––––––
  MethodSummary 216 bytes DataType                                  
  Revise 2.726 MiB Module                                    
  entr 0 bytes entr (generic function with 2 methods)    
  includet 0 bytes includet (generic function with 2 methods)
  revise 0 bytes revise (generic function with 3 methods) 

```

and

```julia
julia> varinfo(MyPackage)
  name size summary
  ––––––– –––––––––– –––––––
  MyPackage 15.182 MiB Module

```

Which seems to be little.

* * *

Strangely, after I go for lunch and return after 30min, the memory usage of the process is now 3.6G. No idea what happened.

* * *

It seems to be related be compiled methods, which uses far more memory than I’ve expect. But I don’t know how to get more details.

What I’ve done:

use top to track the memory usage of julia (1.7.1).  
after starting a new process: ~200M  
after using MyPackage: ~350M  
after executing some precompile commands (~250 precompile commands, many third parties library are used): ~780M  
after GC: ~720M  
some output at this time:

```julia
julia> varinfo()
  name size summary
  –––––––––––––––– ––––––––––– –––––––
  Base Module 
  Core Module 
  InteractiveUtils 253.909 KiB Module 
  Main Module 
  ans 1 byte Bool   

julia> Base.gc_live_bytes()
383446566

```

For solving a real problem, many more methods will be compiled, so in the original problem, maybe more memory are used by compiled codes, but I’ve not find a way to figure out a precise number.

And, after all, it is strange that compiled code should use so many memory. Considering my package is expected to be used with mpi on a cluster with less than 4GB memory per core, this is too high a cost.

---

<div class="post-metadata">

### Author: ![aramirezreyes](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aramirezreyes/32/42573_2.png) [@aramirezreyes](https://discourse.julialang.org/u/aramirezreyes)
#### Post date: [February 20, 2023, 3:50am UTC](https://discourse.julialang.org/t/how-to-check-where-memory-is-used-in-repl/94895/2 "2023-02-20T03:50:04Z")

</div>

It could be that the memory is not taken by any variables but was used by variables that went out of scope. The garbage collector is supposed to recover them eventually but you can force it to act by doing:

```julia
GC.gc()

```

In the REPL and then checking again your memory usage.

---

<div class="post-metadata">

### Author: ![jsjie](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jsjie/32/22477_2.png) [@jsjie](https://discourse.julialang.org/u/jsjie)
#### Post date: [February 20, 2023, 4:12am UTC](https://discourse.julialang.org/t/how-to-check-where-memory-is-used-in-repl/94895/3 "2023-02-20T04:12:51Z")

</div>

The data reported above is after I run `GC.gc()`. It only frees about 80M memory.

---

<div class="post-metadata">

### Author: ![jakobnissen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jakobnissen/32/13477_2.png) [@jakobnissen](https://discourse.julialang.org/u/jakobnissen)
#### Post date: [February 20, 2023, 8:04am UTC](https://discourse.julialang.org/t/how-to-check-where-memory-is-used-in-repl/94895/4 "2023-02-20T08:04:02Z")

</div>

You can try to look at `Base.summarysize` which tries to give the total size of all objects reachable from a given object. Try running it on all the objects inside `varinfo()`:

```julia
julia> x = rand(Float32, 1000, 1000);

julia> s = Dict(i => join(rand('A':'z', 200)) for i in 1:100);

julia> for name in names(Main)
           thing = getproperty(Main, name)
           thing isa Module && continue
           println(String(name), ' ', Base.summarysize(thing))
       end
ans 25336
s 25336
x 4000040

```

There are probably some edge cases I’ve forgotten about. For example, in my case above, `ans` and `s` are the same object, but appears twice in the output.

---

<div class="post-metadata">

### Author: ![jsjie](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jsjie/32/22477_2.png) [@jsjie](https://discourse.julialang.org/u/jsjie)
#### Post date: [February 21, 2023, 1:14am UTC](https://discourse.julialang.org/t/how-to-check-where-memory-is-used-in-repl/94895/5 "2023-02-21T01:14:16Z")

</div>

For the main object mentioned above, `Base.summarysize` also returns about 1.3G.
