# Triggerring GC unexpectedly

**URL:** <https://discourse.julialang.org/t/triggerring-gc-unexpectedly/86754>\
**Category:** New to Julia\
**Created:** [September 4, 2022, 1:04pm UTC](https://discourse.julialang.org/t/triggerring-gc-unexpectedly/86754 "2022-09-04T13:04:03Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![Brian1](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/brian1/32/49250_2.png) [@Brian1](https://discourse.julialang.org/u/Brian1)\
**Post date:** [September 4, 2022, 1:04pm UTC](https://discourse.julialang.org/t/triggerring-gc-unexpectedly/86754/1 "2022-09-04T13:04:03Z")

</div>

I run these codes:

```julia
@time for i = 1 : 50
    _empty!.(dit)
    _update(dit,dil)
    @time distral(calc_ptm)
end

```

`distral(calc_ptm)` is running on multiple threads. Here is the resulte:

```julia
 16.408509 seconds (15.03 M allocations: 1.160 GiB, 87.29% gc time, 8.77% compilation time)
  0.966252 seconds (12.99 M allocations: 1.059 GiB, 6.43% compilation time)
  0.756220 seconds (12.93 M allocations: 1.056 GiB, 12.48% compilation time)
  0.687300 seconds (12.92 M allocations: 1.055 GiB, 4.60% compilation time)
  0.798255 seconds (12.91 M allocations: 1.054 GiB, 12.11% compilation time)
  0.766506 seconds (12.86 M allocations: 1.051 GiB, 6.66% compilation time)
  0.823462 seconds (12.88 M allocations: 1.052 GiB, 11.30% compilation time)
  0.826604 seconds (12.86 M allocations: 1.051 GiB, 1.86% compilation time)
  0.745301 seconds (12.88 M allocations: 1.052 GiB, 4.22% compilation time)
119.144726 seconds (12.90 M allocations: 1.053 GiB, 99.30% gc time, 0.02% compilation time)
  0.769532 seconds (12.87 M allocations: 1.052 GiB, 7.58% compilation time)
  0.776132 seconds (12.86 M allocations: 1.052 GiB, 1.57% compilation time)
  0.863049 seconds (12.87 M allocations: 1.051 GiB, 2.67% compilation time)
  0.719255 seconds (12.86 M allocations: 1.050 GiB, 7.90% compilation time)
  0.761964 seconds (12.87 M allocations: 1.051 GiB, 1.80% compilation time)
  0.794154 seconds (12.87 M allocations: 1.051 GiB, 5.70% compilation time)
  0.744420 seconds (12.88 M allocations: 1.052 GiB, 5.56% compilation time)
  0.791541 seconds (12.87 M allocations: 1.051 GiB, 1.62% compilation time)
  0.767297 seconds (12.86 M allocations: 1.051 GiB, 1.66% compilation time)
  0.774355 seconds (12.85 M allocations: 1.050 GiB, 1.73% compilation time)
  0.752894 seconds (12.85 M allocations: 1.050 GiB, 1.32% compilation time)
  0.819478 seconds (12.84 M allocations: 1.049 GiB, 10.69% compilation time)
  0.871703 seconds (12.84 M allocations: 1.050 GiB, 4.64% compilation time)
  0.879502 seconds (12.87 M allocations: 1.052 GiB, 1.34% compilation time)
  0.754354 seconds (12.86 M allocations: 1.052 GiB, 1.42% compilation time)
  0.813125 seconds (12.86 M allocations: 1.052 GiB, 1.28% compilation time)
  0.861510 seconds (12.87 M allocations: 1.053 GiB, 5.99% compilation time)
  0.775466 seconds (12.87 M allocations: 1.052 GiB, 5.04% compilation time)
  0.721221 seconds (12.85 M allocations: 1.051 GiB, 1.07% compilation time)
  0.882465 seconds (12.85 M allocations: 1.051 GiB, 0.83% compilation time)
  0.849271 seconds (12.88 M allocations: 1.053 GiB, 6.32% compilation time)
  0.875740 seconds (12.88 M allocations: 1.054 GiB, 1.36% compilation time)
1107.545609 seconds (12.90 M allocations: 1.054 GiB, 99.92% gc time, 0.01% compilation time: 91% of which was recompilation)
  0.895698 seconds (12.87 M allocations: 1.052 GiB, 0.91% compilation time)
  0.810025 seconds (12.86 M allocations: 1.052 GiB, 0.92% compilation time)
  0.756983 seconds (12.86 M allocations: 1.052 GiB, 0.99% compilation time)
  0.810946 seconds (12.85 M allocations: 1.052 GiB, 2.12% compilation time)
  0.722169 seconds (12.87 M allocations: 1.053 GiB, 3.36% compilation time)
  0.777227 seconds (12.86 M allocations: 1.052 GiB, 0.93% compilation time)
  0.788414 seconds (12.87 M allocations: 1.052 GiB, 2.88% compilation time)
  0.757612 seconds (12.86 M allocations: 1.052 GiB, 0.73% compilation time)
  0.777249 seconds (12.86 M allocations: 1.052 GiB, 0.65% compilation time)
  0.797138 seconds (12.86 M allocations: 1.053 GiB, 0.62% compilation time)
  0.773889 seconds (12.86 M allocations: 1.054 GiB, 0.77% compilation time)
  0.947090 seconds (12.88 M allocations: 1.054 GiB, 0.80% compilation time)
267.207840 seconds (12.89 M allocations: 1.055 GiB, 99.72% gc time, 0.03% compilation time)
  0.832947 seconds (12.88 M allocations: 1.054 GiB, 1.13% compilation time)
  0.821811 seconds (12.88 M allocations: 1.054 GiB, 0.79% compilation time)
  0.766192 seconds (12.88 M allocations: 1.054 GiB, 1.47% compilation time)
  0.799973 seconds (12.87 M allocations: 1.054 GiB, 0.71% compilation time)
1547.545871 seconds (646.22 M allocations: 52.749 GiB, 97.30% gc time, 0.19% compilation time: 3% of which was recompilation)

```

As can be seen above, there are some iters in the loop have many gc time. How can I avoid the unexpectly huge gc time ? I tried:

```julia
GC.enable(false)
@time for i = 1 : 50
    _empty!.(dit)
    _update(dit,dil)
    @time distral(calc_ptm)
end
GC.enable(true)

```

But then, `GC.gc()` is always in excueting, and seems never stop.

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [September 4, 2022, 1:06pm UTC](https://discourse.julialang.org/t/triggerring-gc-unexpectedly/86754/2 "2022-09-04T13:06:54Z")

</div>

you have so much allocation what do you want to happen if GC is disabled, crashing the computer?

---

<div class="post-metadata">

**Author:** ![ufechner7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ufechner7/32/51363_2.png) [@ufechner7](https://discourse.julialang.org/u/ufechner7)\
**Post date:** [September 4, 2022, 1:14pm UTC](https://discourse.julialang.org/t/triggerring-gc-unexpectedly/86754/3 "2022-09-04T13:14:58Z")

</div>

Perhaps try to understand where the allocations are coming from and reducing them as first step?

Try:

- preallocating arrays
- use the dot operator
- use views

---

<div class="post-metadata">

**Author:** ![Brian1](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/brian1/32/49250_2.png) [@Brian1](https://discourse.julialang.org/u/Brian1)\
**Post date:** [September 4, 2022, 1:58pm UTC](https://discourse.julialang.org/t/triggerring-gc-unexpectedly/86754/4 "2022-09-04T13:58:35Z")

</div>

Yes, the allocation is huge. I supposed to gc manually, turning off gc before running that block codes, and after that turning on gc. Mabe it is an normal operation? But after I turn gc on again, `GC.gc()` never stop.

---

<div class="post-metadata">

**Author:** ![Brian1](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/brian1/32/49250_2.png) [@Brian1](https://discourse.julialang.org/u/Brian1)\
**Post date:** [September 4, 2022, 2:04pm UTC](https://discourse.julialang.org/t/triggerring-gc-unexpectedly/86754/5 "2022-09-04T14:04:11Z")

</div>

I applied all the performace tips calfully on my codes, and there is really little room to reduce allocation further. The RAM on my computer is 128G, I do not need gc on that frequency. So I seek to gc maually, but it seems not working. Maybe there is some critical attention I need to pay on `GC.enable()`?

---

<div class="post-metadata">

**Author:** ![Brian1](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/brian1/32/49250_2.png) [@Brian1](https://discourse.julialang.org/u/Brian1)\
**Post date:** [September 4, 2022, 2:08pm UTC](https://discourse.julialang.org/t/triggerring-gc-unexpectedly/86754/6 "2022-09-04T14:08:53Z")

</div>

The critical point is, the garbage produed by multithreads is really slow the remove by `GC.gc()` on the main thread. Maybe it is the reason that `GC.gc()` nerver stop?

---

<div class="post-metadata">

**Author:** ![SteffenPL](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/steffenpl/32/206270_2.png) [@SteffenPL](https://discourse.julialang.org/u/SteffenPL)\
**Post date:** [September 4, 2022, 2:53pm UTC](https://discourse.julialang.org/t/triggerring-gc-unexpectedly/86754/7 "2022-09-04T14:53:39Z")

</div>

Did you use profilers (such as [Profiler · Julia in VS Code](https://www.julia-vscode.org/docs/dev/userguide/profiler/) ) to detect which parts of the program need most attention?

For a function which runs in less than a second it is quite likely that 12M allocation calls are too much. I would also recommend to try to reduce the allocations and to figure out which part of the code allocates.  
It’s not just a question about available RAM, but even with enough RAM the allocations probably  
indicate that somewhere the code is inefficient.

(I know, it’s not the answer you are looking for, but I can tell you that one year ago I was actually in a similar situation; also having tons of allocations and little hope that I could  
get rid of them. But in fact it was only a very innocent function which allocated, so, profiling was really the key since before all my focus was on the wrong parts of the code.)

---

<div class="post-metadata">

**Author:** ![chf](https://avatars.discourse-cdn.com/v4/letter/c/f14d63/32.png) [@chf](https://discourse.julialang.org/u/chf)\
**Post date:** [September 4, 2022, 3:22pm UTC](https://discourse.julialang.org/t/triggerring-gc-unexpectedly/86754/8 "2022-09-04T15:22:25Z")

</div>

I made a change which factors in the available RAM when deciding when to GC. Basically we put off doing a full GC until you absolutely need to (that’s the big pauses you are seeing). Can you try running your program on the latest development build?

---

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [September 4, 2022, 10:41pm UTC](https://discourse.julialang.org/t/triggerring-gc-unexpectedly/86754/9 "2022-09-04T22:41:34Z")

</div>

> [@Brian1](#):
>
> The RAM on my computer is 128G, I do not need gc on that frequency.

That’s bad thinking. It’s dangerous to postpone GC in case you have enough RAM. You’re not reducing the work, just postponing the work, and it will be that much larger later.

On the contrary, a beter approach is more frequent GC (if you can not do fewer allocations). There’s a new option in Julia 1.9:

> New option `--heap-size-hint=<size>` gives a memory hint for triggering greedy garbage collection.

That would always be safe, and you can try to set it low to trigger more GC. It’s one of the few options (one only) to tune. There’s some other work on the GC that’s not yet in, or some may be, I would try out master/1.9.

---

<div class="post-metadata">

**Author:** ![ufechner7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ufechner7/32/51363_2.png) [@ufechner7](https://discourse.julialang.org/u/ufechner7)\
**Post date:** [September 5, 2022, 4:45am UTC](https://discourse.julialang.org/t/triggerring-gc-unexpectedly/86754/10 "2022-09-05T04:45:05Z")

</div>

In some of my code I do a frequent, incremental GC 20 times per second, see [KiteSimulators.jl/autopilot.jl at main · aenarete/KiteSimulators.jl · GitHub](https://github.com/aenarete/KiteSimulators.jl/blob/main/examples/autopilot.jl)

```julia
t_gc_tot += @elapsed GC.gc(false)

```

Line 56  
This allows me to run my code in real-time most of the time. And I am monitoring the GC time to see if code changes improve it or make it worse.