Triggerring GC unexpectedly

New to Julia
1

I run these codes:

@time for i = 1 : 50
    _empty!.(dit)
    _update(dit,dil)
    @time distral(calc_ptm)
end

distral(calc_ptm) is running on multiple threads. Here is the resulte:

 16.408509 seconds (15.03 M allocations: 1.160 GiB, 87.29% gc time, 8.77% compilation time)
  0.966252 seconds (12.99 M allocations: 1.059 GiB, 6.43% compilation time)
  0.756220 seconds (12.93 M allocations: 1.056 GiB, 12.48% compilation time)
  0.687300 seconds (12.92 M allocations: 1.055 GiB, 4.60% compilation time)
  0.798255 seconds (12.91 M allocations: 1.054 GiB, 12.11% compilation time)
  0.766506 seconds (12.86 M allocations: 1.051 GiB, 6.66% compilation time)
  0.823462 seconds (12.88 M allocations: 1.052 GiB, 11.30% compilation time)
  0.826604 seconds (12.86 M allocations: 1.051 GiB, 1.86% compilation time)
  0.745301 seconds (12.88 M allocations: 1.052 GiB, 4.22% compilation time)
119.144726 seconds (12.90 M allocations: 1.053 GiB, 99.30% gc time, 0.02% compilation time)
  0.769532 seconds (12.87 M allocations: 1.052 GiB, 7.58% compilation time)
  0.776132 seconds (12.86 M allocations: 1.052 GiB, 1.57% compilation time)
  0.863049 seconds (12.87 M allocations: 1.051 GiB, 2.67% compilation time)
  0.719255 seconds (12.86 M allocations: 1.050 GiB, 7.90% compilation time)
  0.761964 seconds (12.87 M allocations: 1.051 GiB, 1.80% compilation time)
  0.794154 seconds (12.87 M allocations: 1.051 GiB, 5.70% compilation time)
  0.744420 seconds (12.88 M allocations: 1.052 GiB, 5.56% compilation time)
  0.791541 seconds (12.87 M allocations: 1.051 GiB, 1.62% compilation time)
  0.767297 seconds (12.86 M allocations: 1.051 GiB, 1.66% compilation time)
  0.774355 seconds (12.85 M allocations: 1.050 GiB, 1.73% compilation time)
  0.752894 seconds (12.85 M allocations: 1.050 GiB, 1.32% compilation time)
  0.819478 seconds (12.84 M allocations: 1.049 GiB, 10.69% compilation time)
  0.871703 seconds (12.84 M allocations: 1.050 GiB, 4.64% compilation time)
  0.879502 seconds (12.87 M allocations: 1.052 GiB, 1.34% compilation time)
  0.754354 seconds (12.86 M allocations: 1.052 GiB, 1.42% compilation time)
  0.813125 seconds (12.86 M allocations: 1.052 GiB, 1.28% compilation time)
  0.861510 seconds (12.87 M allocations: 1.053 GiB, 5.99% compilation time)
  0.775466 seconds (12.87 M allocations: 1.052 GiB, 5.04% compilation time)
  0.721221 seconds (12.85 M allocations: 1.051 GiB, 1.07% compilation time)
  0.882465 seconds (12.85 M allocations: 1.051 GiB, 0.83% compilation time)
  0.849271 seconds (12.88 M allocations: 1.053 GiB, 6.32% compilation time)
  0.875740 seconds (12.88 M allocations: 1.054 GiB, 1.36% compilation time)
1107.545609 seconds (12.90 M allocations: 1.054 GiB, 99.92% gc time, 0.01% compilation time: 91% of which was recompilation)
  0.895698 seconds (12.87 M allocations: 1.052 GiB, 0.91% compilation time)
  0.810025 seconds (12.86 M allocations: 1.052 GiB, 0.92% compilation time)
  0.756983 seconds (12.86 M allocations: 1.052 GiB, 0.99% compilation time)
  0.810946 seconds (12.85 M allocations: 1.052 GiB, 2.12% compilation time)
  0.722169 seconds (12.87 M allocations: 1.053 GiB, 3.36% compilation time)
  0.777227 seconds (12.86 M allocations: 1.052 GiB, 0.93% compilation time)
  0.788414 seconds (12.87 M allocations: 1.052 GiB, 2.88% compilation time)
  0.757612 seconds (12.86 M allocations: 1.052 GiB, 0.73% compilation time)
  0.777249 seconds (12.86 M allocations: 1.052 GiB, 0.65% compilation time)
  0.797138 seconds (12.86 M allocations: 1.053 GiB, 0.62% compilation time)
  0.773889 seconds (12.86 M allocations: 1.054 GiB, 0.77% compilation time)
  0.947090 seconds (12.88 M allocations: 1.054 GiB, 0.80% compilation time)
267.207840 seconds (12.89 M allocations: 1.055 GiB, 99.72% gc time, 0.03% compilation time)
  0.832947 seconds (12.88 M allocations: 1.054 GiB, 1.13% compilation time)
  0.821811 seconds (12.88 M allocations: 1.054 GiB, 0.79% compilation time)
  0.766192 seconds (12.88 M allocations: 1.054 GiB, 1.47% compilation time)
  0.799973 seconds (12.87 M allocations: 1.054 GiB, 0.71% compilation time)
1547.545871 seconds (646.22 M allocations: 52.749 GiB, 97.30% gc time, 0.19% compilation time: 3% of which was recompilation)

As can be seen above, there are some iters in the loop have many gc time. How can I avoid the unexpectly huge gc time ? I tried:

GC.enable(false)
@time for i = 1 : 50
    _empty!.(dit)
    _update(dit,dil)
    @time distral(calc_ptm)
end
GC.enable(true)

But then, GC.gc() is always in excueting, and seems never stop.

2

you have so much allocation what do you want to happen if GC is disabled, crashing the computer?

3

Perhaps try to understand where the allocations are coming from and reducing them as first step?

Try:

  • preallocating arrays
  • use the dot operator
  • use views
1 Like
4

Yes, the allocation is huge. I supposed to gc manually, turning off gc before running that block codes, and after that turning on gc. Mabe it is an normal operation? But after I turn gc on again, GC.gc() never stop.

5

I applied all the performace tips calfully on my codes, and there is really little room to reduce allocation further. The RAM on my computer is 128G, I do not need gc on that frequency. So I seek to gc maually, but it seems not working. Maybe there is some critical attention I need to pay on GC.enable()?

6

The critical point is, the garbage produed by multithreads is really slow the remove by GC.gc() on the main thread. Maybe it is the reason that GC.gc() nerver stop?

7

Did you use profilers (such as Profiler · Julia in VS Code ) to detect which parts of the program need most attention?

For a function which runs in less than a second it is quite likely that 12M allocation calls are too much. I would also recommend to try to reduce the allocations and to figure out which part of the code allocates.
It’s not just a question about available RAM, but even with enough RAM the allocations probably
indicate that somewhere the code is inefficient.

(I know, it’s not the answer you are looking for, but I can tell you that one year ago I was actually in a similar situation; also having tons of allocations and little hope that I could
get rid of them. But in fact it was only a very innocent function which allocated, so, profiling was really the key since before all my focus was on the wrong parts of the code.)

8

I made a change which factors in the available RAM when deciding when to GC. Basically we put off doing a full GC until you absolutely need to (that’s the big pauses you are seeing). Can you try running your program on the latest development build?

9

That’s bad thinking. It’s dangerous to postpone GC in case you have enough RAM. You’re not reducing the work, just postponing the work, and it will be that much larger later.

On the contrary, a beter approach is more frequent GC (if you can not do fewer allocations). There’s a new option in Julia 1.9:

New option --heap-size-hint=<size> gives a memory hint for triggering greedy garbage collection.

That would always be safe, and you can try to set it low to trigger more GC. It’s one of the few options (one only) to tune. There’s some other work on the GC that’s not yet in, or some may be, I would try out master/1.9.

10

In some of my code I do a frequent, incremental GC 20 times per second, see KiteSimulators.jl/autopilot.jl at main · aenarete/KiteSimulators.jl · GitHub

t_gc_tot += @elapsed GC.gc(false)

Line 56
This allows me to run my code in real-time most of the time. And I am monitoring the GC time to see if code changes improve it or make it worse.