# Memory sizing and GC behavior

**URL:** <https://discourse.julialang.org/t/memory-sizing-and-gc-behavior/10121>\
**Category:** General Usage\
**Created:** [April 2, 2018, 6:27pm UTC](https://discourse.julialang.org/t/memory-sizing-and-gc-behavior/10121 "2018-04-02T18:27:31Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![tk3369](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tk3369/32/2824_2.png) [@tk3369](https://discourse.julialang.org/u/tk3369)\
**Post date:** [April 2, 2018, 6:27pm UTC](https://discourse.julialang.org/t/memory-sizing-and-gc-behavior/10121/1 "2018-04-02T18:27:31Z")

</div>

I have a function that allocates a lot and hence every trial requires some time for GC. I’m just wondering if there’s any way to tune the system such that it has more heap memory and avoid having to GC so often?

```julia
BenchmarkTools.Trial:
  memory estimate: 852.61 MiB
  allocs estimate: 59101
  --------------
  minimum time: 1.566 s (35.72% GC)
  median time: 1.609 s (35.45% GC)
  mean time: 1.609 s (35.82% GC)
  maximum time: 1.754 s (40.24% GC)
  --------------
  samples: 19
  evals/sample: 1

```

---

<div class="post-metadata">

**Author:** ![rdeits](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rdeits/32/286_2.png) [@rdeits](https://discourse.julialang.org/u/rdeits)\
**Post date:** [April 2, 2018, 6:30pm UTC](https://discourse.julialang.org/t/memory-sizing-and-gc-behavior/10121/2 "2018-04-02T18:30:07Z")

</div>

You can disable the garbage collector for a particular block of code with `gc_enable(false)` and then re-enable it with `gc_enable(true)` and you can even force garbage collection with `gc()`. That said, I’ve not personally found any cases where doing so actually helped my code.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [April 3, 2018, 5:47am UTC](https://discourse.julialang.org/t/memory-sizing-and-gc-behavior/10121/3 "2018-04-03T05:47:17Z")

</div>

Allocations are _may_ be an indicator of suboptimal code, so that it is worth [optimizing](https://docs.julialang.org/en/latest/manual/performance-tips/#Measure-performance-with-%5B@time%5D(@ref)-and-pay-attention-to-memory-allocation-1) in itself, but if some allocation is inherent to the algorithm, you can’t do much about it.

It is hard to say more without an MWE, but I would try `StaticArrays` (first choice) or pre-allocated buffers if possible.

---

<div class="post-metadata">

**Author:** ![tk3369](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tk3369/32/2824_2.png) [@tk3369](https://discourse.julialang.org/u/tk3369)\
**Post date:** [April 3, 2018, 7:11am UTC](https://discourse.julialang.org/t/memory-sizing-and-gc-behavior/10121/4 "2018-04-03T07:11:52Z")

</div>

@rdeits If I disable GC completely then wouldn’t it be possible to run out of system memory causing a crash?

@Tamas_Papp I have a very large SharedArray that totals about 30 GiB in size. I need to take a subset of the matrix (up to half of its size) and perform a series of matrix computation. I could use a view but it makes the computation slower. Pre-allocation is a decent idea but I would have to wrap it to support variable sizes. Know any good package for that?

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [April 3, 2018, 7:20am UTC](https://discourse.julialang.org/t/memory-sizing-and-gc-behavior/10121/5 "2018-04-03T07:20:42Z")

</div>

> [@tk3369](#):
>
> Pre-allocation is a decent idea but I would have to wrap it to support variable sizes. Know any good package for that?

No. In my experience coding these directly is the path of least resistance.

(For 30 GiB objects, please disregard the suggestion for `StaticArrays` 😄)

---

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [April 3, 2018, 12:31pm UTC](https://discourse.julialang.org/t/memory-sizing-and-gc-behavior/10121/6 "2018-04-03T12:31:36Z")

</div>

> [@tk3369](#):
>
> If I disable GC completely then wouldn’t it be possible to run out of system memory causing a crash?

Yes, and what is worse this could be slower, at least in theory. [Slower, as in throughput; I would only consider doing this in soft-real-time situations, and then performance may be more even, but I’m not really convinced of that either for the reason below. Best would be to get rid of [heap] allocations in that case, and then GC isn’t a problem and needs not be disabled.]

Julia’s GC is generational. With the GC on, you could be freeing memory in the younger generation that is likely to be in cache, so new allocations should go there. Disabling GC, you have no choice other than allocating more and more memory, eventually allocating more than fits in cache; this would happen before running out of memory, but would make your program slower.

---

<div class="post-metadata">

**Author:** ![Samuel\_Ainsworth](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/samuel_ainsworth/32/15644_2.png) [@Samuel\_Ainsworth](https://discourse.julialang.org/u/Samuel_Ainsworth)\
**Post date:** [July 23, 2020, 2:01am UTC](https://discourse.julialang.org/t/memory-sizing-and-gc-behavior/10121/7 "2020-07-23T02:01:01Z")

</div>

Is there a way to adjust the heap size? I know that other GCs offer various knobs to adjust performance, eg. Java. I’m currently running a script that stops for GC constantly and is spending 20% of its time in GC, even though it’s on a machine with 128Gb of RAM so I could conceivably run much longer periods before stopping for GC.