# Where is my memory usage coming from

**URL:** <https://discourse.julialang.org/t/where-is-my-memory-usage-coming-from/67351>\
**Category:** General Usage\
**Tags:** garbage-collection\
**Created:** [August 30, 2021, 1:27pm UTC](https://discourse.julialang.org/t/where-is-my-memory-usage-coming-from/67351 "2021-08-30T13:27:53Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![maarten\_van\_damme](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maarten_van_damme/32/6946_2.png) [@maarten\_van\_damme](https://discourse.julialang.org/u/maarten_van_damme)\
**Post date:** [August 30, 2021, 1:27pm UTC](https://discourse.julialang.org/t/where-is-my-memory-usage-coming-from/67351/1 "2021-08-30T13:27:54Z")

</div>

While developing a package, I noticed that a simulation that used to run without problems now takes a whopping 30+ gb. The script is rather simple, I use some packages and then execute a let block. There should be no leaked variables into global scope, as everything is enclosed within the let block. Yet at the end of the simulation, julia keeps taking up this amount of memory. The usual way to analyze allocations is to track them, but I’m more interested why they aren’t being garbage collected at the end of the simulation (even after GC.gc())

The packages it depend on do use some kind of caching, and the summarysizes of those caches add up to about 600 megabytes.

[Here](https://gist.github.com/maartenvd/4c429d5cf11ad9d12dd66e6bc8317074) is the script in question (but I have simplified the simulation to be runable on my laptop, but still with way too high memory usage). To run it, one needs

[this branch of MPSKit](https://github.com/maartenvd/MPSKit.jl/tree/planocalypse)

[this fork of TensorKit](https://github.com/maartenvd/TensorKit.jl)

I’m primarily interested in the memory usage after the simulation is done, as I at the very least expect this to be manageable.

---

<div class="post-metadata">

**Author:** ![maarten\_van\_damme](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maarten_van_damme/32/6946_2.png) [@maarten\_van\_damme](https://discourse.julialang.org/u/maarten_van_damme)\
**Post date:** [September 1, 2021, 5:45am UTC](https://discourse.julialang.org/t/where-is-my-memory-usage-coming-from/67351/2 "2021-09-01T05:45:13Z")

</div>

I think I know where the memory is being used for, but I don’t understand why summarysize does not show the proper amount of memory used:

```julia
julia> total = 0; for (k,v) in SUNRepresentations.CGCCACHE[(3,Float64)]
       total+=Base.summarysize(v)
       end

julia> total
2184221472

julia> Base.summarysize(SUNRepresentations.CGCCACHE[(3,Float64)])
463032

```

There are no duplicates in that cache, every key has a unique value - counterpart.

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [September 1, 2021, 7:13am UTC](https://discourse.julialang.org/t/where-is-my-memory-usage-coming-from/67351/3 "2021-09-01T07:13:58Z")

</div>

[https://github.com/maartenvd/SUNRepresentations.jl/blob/8a49c041f8d7f69090b54588dd0eb50c081ef0bc/src/clebschgordan.jl#L17](https://github.com/maartenvd/SUNRepresentations.jl/blob/8a49c041f8d7f69090b54588dd0eb50c081ef0bc/src/clebschgordan.jl#L17)

The cache is a non-constant global `Dict{Any,Any}` - basically anything about this is bound to be slow & hard to clean up. The compiler has to check both type and location of that global every time it is accessed since it may change at any time. Accessing global variables may also allocate:

```julia
julia> a = rand(1000);                      
                                            
julia> f(x) = sum(a) * x                    
f (generic function with 1 method)          
                                            
julia> @time f(5)                           
  0.000008 seconds (2 allocations: 32 bytes)
2500.5146450452835                          

```

What kind of data are you storing in that `Dict` that requires it to be typed `Any`?

---

<div class="post-metadata">

**Author:** ![maarten\_van\_damme](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maarten_van_damme/32/6946_2.png) [@maarten\_van\_damme](https://discourse.julialang.org/u/maarten_van_damme)\
**Post date:** [September 1, 2021, 7:30am UTC](https://discourse.julialang.org/t/where-is-my-memory-usage-coming-from/67351/4 "2021-09-01T07:30:42Z")

</div>

this cache is actually a dict of dicts. The problem is that you may want to cache the clebsch gordon coefficients of SU(N), for multiple N’s. And then you may want this object at multiple levels of precision (so different eltypes). The dictionary is therefore (element\_type,N) =\> Dictionary of (SUN sectors) =\> the clebsch gordon coefficients.

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [September 1, 2021, 7:41am UTC](https://discourse.julialang.org/t/where-is-my-memory-usage-coming-from/67351/5 "2021-09-01T07:41:10Z")

</div>

Right. So the type of the keys of `CGCCACHE` seems to be `Tuple{DataType,Int}`and the values are of type `Dict{NTuple{nSun, SUNIrrep{N}}, SparseArray{T,4}} where {N, nSUN, T}`.

Neither seems to require `Dict{Any,Any}` nor global constants - why not have a `constructCache()` function that gives the user a cache to manage themselves? That would prevent unwanted memory growth after execution since the user can put `constructCache()` into a function just as well, which would allow GC to clean up after the function has exited (or let the user keep the cache around for longer, by returning it again).

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [September 1, 2021, 7:45am UTC](https://discourse.julialang.org/t/where-is-my-memory-usage-coming-from/67351/6 "2021-09-01T07:45:01Z")

</div>

> [@Sukera](#):
>
> Dict{NTuple{nSUN, SUNIrrep{N}}, SparseArray{T,4}} where {N, nSUN, T}

Looking at the code that seems to fill `CGCCACHE`, looks like this type is just

```julia
Dict{NTuple{3, SUNIrrep{N}}, SparseArray{T,4}} where {N, T}

```

instead?

---

<div class="post-metadata">

**Author:** ![maarten\_van\_damme](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maarten_van_damme/32/6946_2.png) [@maarten\_van\_damme](https://discourse.julialang.org/u/maarten_van_damme)\
**Post date:** [September 1, 2021, 7:47am UTC](https://discourse.julialang.org/t/where-is-my-memory-usage-coming-from/67351/7 "2021-09-01T07:47:16Z")

</div>

Right, I mistakenly thought ` typeof(Float64) == Type{Float64}`. I’m not entirely sure how to make a user managed cache play nice with TensorKit, with which this package is primarily meant to be used with.

I guess my confusion was that varinfo(all=true,imported=true,recursive=true) just crashes, and if you then fix this, the summarysizes only add up to a fraction of the total cost (apparantly this is now fixed on v1.7.0 beta 4 [https://github.com/JuliaLang/julia/issues/41941](https://github.com/JuliaLang/julia/issues/41941) )

---

<div class="post-metadata">

**Author:** ![maarten\_van\_damme](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maarten_van_damme/32/6946_2.png) [@maarten\_van\_damme](https://discourse.julialang.org/u/maarten_van_damme)\
**Post date:** [September 1, 2021, 7:49am UTC](https://discourse.julialang.org/t/where-is-my-memory-usage-coming-from/67351/8 "2021-09-01T07:49:22Z")

</div>

There are often cases where I’d like to cache these things to disk instead of keeping old references in ram. Is there some canonical mechanism for this, like how you’re supposed to use the artifact system when working with binary dependencies?

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [September 1, 2021, 8:06am UTC](https://discourse.julialang.org/t/where-is-my-memory-usage-coming-from/67351/9 "2021-09-01T08:06:22Z")

</div>

> [@maarten\_van\_damme](#):
>
> I’m not entirely sure how to make a user managed cache play nice with TensorKit, with which this package is primarily meant to be used with.

I don’t know how TensorKit caches things (if at all, a quick search on github for “cache” in that repo didn’t turn up anything), so I’m not sure what you mean by that. A cursory look through TensorKit didn’t help in figuring out where the memory is spent - the library is a tad too big to just jump in without an explicit (somewhat short) reproducible example of high memory usage.

> [@maarten\_van\_damme](#):
>
> There are often cases where I’d like to cache these things to disk instead of keeping old references in ram. Is there some canonical mechanism for this, like how you’re supposed to use the artifact system when working with binary dependencies?

The artifact system can be used for anything that’s a blob of data to be served to users - it’s not limited to libraries and code. See [here](https://pkgdocs.julialang.org/dev/artifacts/) for more info. How you want to store the data exactly is up to you though and depending on what exactly you want to do/save, you’ll have to choose different serialization strategies.

* * *

Personally, I’m not sure why you’d want to have one cache for all element types. Intuitively it would be better to stay in one element type when doing a calculation/operation.

May I ask, do you want to introduce a cache because TensorKit is highly optimized already, but still too slow, or are you trying to speed up your own calculation by reusing results, in spite of possibly greater gains being made from improving TensorKit?

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [September 1, 2021, 8:20am UTC](https://discourse.julialang.org/t/where-is-my-memory-usage-coming-from/67351/10 "2021-09-01T08:20:14Z")

</div>

> [@Sukera](#):
>
> I don’t know how TensorKit caches things (if at all, a quick search on github for “cache” in that repo didn’t turn up anything), so I’m not sure what you mean by that.

Nevermind, I found them:

```julia
$ rg "cache ="               
src/fusiontrees/manipulations.jl                          
479:const transposecache = LRU{Any, Any}(; maxsize = 10^5)
480:const usetransposecache = Ref{Bool}(true)             
962:const braidcache = LRU{Any, Any}(; maxsize = 10^5)    

```

yeah… those caches have the exact same problem of being globals and `Any` keys and `Any` values. That _will_ kill overall performance and make it much harder to profile…

---

<div class="post-metadata">

**Author:** ![maarten\_van\_damme](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maarten_van_damme/32/6946_2.png) [@maarten\_van\_damme](https://discourse.julialang.org/u/maarten_van_damme)\
**Post date:** [September 1, 2021, 8:25am UTC](https://discourse.julialang.org/t/where-is-my-memory-usage-coming-from/67351/11 "2021-09-01T08:25:20Z")

</div>

I should elaborate. My own calculations require tensorcontractions of SU(3) symmetric tensors. TensorKit works independently of the particular symmetry, you just define what kind of irreps exist for you group along with some properties (one of which is the f-symbols), and of you go.

Tensorkit in turn queries certain f symbols for your irreps, which in this case can be calculated as a function of clebsch gordon coefficients (CGCs). Some f symbols re-use the same CGCs, so it makes sense to store them, in case they have to be re-used.

TensorKit on its own is fast (those caches may be `Any,Any` but all practical cost sits in matrix matrix multiplications), as is sunrepresentations (it’s much faster then the original c++ code from the paper it’s based on), but the combination of the two would be very slow if it has to re-calculated all relevant CGCs every time a new f-symbol is required. This calculation is expensive, much more expensive than a query from a global type unstable cache.

That’s why a user-managed cache also doesn’t entirely fit. As an example, this is the usual way you’d want to use the two:

```julia

tensor = TensorMap(rand #= initializer =#,
    ComplexF64 #= element type of that tensor=#,
    Rep[SU{3}]((0, 0, 0)=>1)#=codomain=#,
    Rep[SU{3}]((0, 0, 0)=>1) #=domain=#);
# ... calculations with tensor

```

There seems to me to be no clean way to use a user managed cache, and I don’t think you’d want to either. I do agree that you really don’t want some kind of object that can blow up indefinitely though (so I should use some kind of LRUcache).

In any case, the original problem seems solved. The summarysize of my caches was wrong, and this is fixed in the latest julia versions.

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [September 1, 2021, 8:54am UTC](https://discourse.julialang.org/t/where-is-my-memory-usage-coming-from/67351/12 "2021-09-01T08:54:10Z")

</div>

> [@maarten\_van\_damme](#):
>
> (those caches may be `Any,Any` but all practical cost sits in matrix matrix multiplications)

How big are these matrices usually? As far as I can tell, TensorKit doesn’t have matrices (as in, multidimensional arrays of values) but operates on the abstract transformations represented by them. Can you point me in the right direction of the code? I’d like to investigate where the high memory use is coming from in the first place.

> [@maarten\_van\_damme](#):
>
> In any case, the original problem seems solved. The summarysize of my caches was wrong, and this is fixed in the latest julia versions.

Well, your question about `summarysize` is solved, but the other part (“why is there so much memory used in the first place”) still stands imo.

* * *

In my experience, having a type unstable frontend can mask problems further down the stack, because the type instability can prevent some optimizations from taking place if it propagates far enough (I haven’t checked with `@code_warntype` though).

---

<div class="post-metadata">

**Author:** ![maarten\_van\_damme](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maarten_van_damme/32/6946_2.png) [@maarten\_van\_damme](https://discourse.julialang.org/u/maarten_van_damme)\
**Post date:** [September 1, 2021, 9:33am UTC](https://discourse.julialang.org/t/where-is-my-memory-usage-coming-from/67351/13 "2021-09-01T09:33:31Z")

</div>

I think know why so much memory is suddenly used. The answer is a bit technical, but the point is that some symmetries don’t allow you to “braid” different spaces. In array terminology, you are no longer allowed to permutedims(A,(2,1,3,4,5)), only cyclic permutations are allowed. I rewrote mpskit so that it indeed only used those kind of operations, but it turns out that this translates to many more f-symbols being required. The dimension of an irrep very quickly blows up if you use SU(3) (the dimension of SU3 (15,7,0) is 612, the CGCs are maps from (irrep,irrep) to irrep.

Tensorkit’s aray operations start out in the tensormap type [https://github.com/Jutho/TensorKit.jl/blob/master/src/tensors/tensor.jl#L10](https://github.com/Jutho/TensorKit.jl/blob/master/src/tensors/tensor.jl#L10) .