# GC occurs at the worst time in tight loop (Garbage Collection)

**URL:** <https://discourse.julialang.org/t/gc-occurs-at-the-worst-time-in-tight-loop-garbage-collection/104934>\
**Category:** Performance\
**Tags:** question\
**Created:** [October 13, 2023, 4:45pm UTC](https://discourse.julialang.org/t/gc-occurs-at-the-worst-time-in-tight-loop-garbage-collection/104934 "2023-10-13T16:45:17Z")\
**Posts on this page:** 14\
**Page:** 5

<div class="post-metadata">

**Author:** ![Allan\_Baker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/allan_baker/32/42645_2.png) [@Allan\_Baker](https://discourse.julialang.org/u/Allan_Baker)\
**Post date:** [October 26, 2023, 8:04pm UTC](https://discourse.julialang.org/t/gc-occurs-at-the-worst-time-in-tight-loop-garbage-collection/104934/81 "2023-10-26T20:04:49Z")

</div>

```julia
julia> @code_llvm x.rfhwil_ccall_interface.julia_set_dB_gain(1.0)
; @ C:\mypath\src\rfhwil_ccall_interface.jl:375 within `julia_set_dB_gain`
; Function Attrs: uwtable
define void @julia_julia_set_dB_gain_1762(double %0) #0 {
top:
; @ C:\mypath\src\rfhwil_ccall_interface.jl:376 within `julia_set_dB_gain`
; ┌ @ refvalue.jl:57 within `setindex!`
; │┌ @ Base.jl:38 within `setproperty!`
    store i8 0, i8* inttoptr (i64 140710129657280 to i8*), align 64
; └└
; @ C:\mypath\src\rfhwil_ccall_interface.jl:377 within `julia_set_dB_gain`
; ┌ @ refvalue.jl:56 within `getindex`
; │┌ @ Base.jl:37 within `getproperty`
    %1 = load atomic {}*, {}**inttoptr (i64 140710129665216 to {}**) unordered, align 64
    %.not = icmp eq {}* %1, null
    br i1 %.not, label %fail, label %pass

fail: ; preds = %top
    call void @ijl_throw({}* inttoptr (i64 140710407392992 to {}*))
    unreachable

pass: ; preds = %top
; └└
; ┌ @ Base.jl:38 within `setproperty!`
   %2 = bitcast {}* %1 to i8*
   %3 = getelementptr inbounds i8, i8* %2, i64 248
   %4 = bitcast i8* %3 to double*
   store double %0, double* %4, align 8
; └
; @ C:\mypath\src\rfhwil_ccall_interface.jl:378 within `julia_set_dB_gain`
  ret void
}

```

---

<div class="post-metadata">

**Author:** ![vchuravy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/vchuravy/32/8_2.png) [@vchuravy](https://discourse.julialang.org/u/vchuravy)\
**Post date:** [October 26, 2023, 8:16pm UTC](https://discourse.julialang.org/t/gc-occurs-at-the-worst-time-in-tight-loop-garbage-collection/104934/82 "2023-10-26T20:16:03Z")

</div>

```julia
Base.@ccallable function julia_set_dB_gain(dB_gain::Cdouble)::Cvoid 
    isCalculationValid[]=false # Parameter Set, so invalidate current answer
    inputRef[].dB_gain = dB_gain
    return nothing
end

f(x) = ccall("extern julia_set_dB_gain", llvmcall, Cvoid, (Cdouble,), x)

using BenchmarkTools

julia> @benchmark f(1.0)
BenchmarkTools.Trial: 10000 samples with 998 evaluations.
 Range (min … max): 16.092 ns … 230.391 ns ┊ GC (min … max): 0.00% … 83.13%
 Time (median): 17.305 ns ┊ GC (median): 0.00%
 Time (mean ± σ): 18.390 ns ± 6.985 ns ┊ GC (mean ± σ): 0.48% ± 2.21%

  ▂ ▆█▄▃▁▅ ▁▃▂▁▁ ▁ ▂
  █▅▁▄▅▆███████▅▅▇▆▃▃▄▁▁▃▃▄▄▅▄▄▅▆█▇▇▆▅▅▄▄██████▇▆▅▄▅███▇▇▆▅▄▁▃ █
  16.1 ns Histogram: log(frequency) by time 26.1 ns <

 Memory estimate: 16 bytes, allocs estimate: 1.

```

That surprised me, but it seems the argument passing is indeed the culprit…

---

<div class="post-metadata">

**Author:** ![vchuravy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/vchuravy/32/8_2.png) [@vchuravy](https://discourse.julialang.org/u/vchuravy)\
**Post date:** [October 26, 2023, 9:01pm UTC](https://discourse.julialang.org/t/gc-occurs-at-the-worst-time-in-tight-loop-garbage-collection/104934/83 "2023-10-26T21:01:12Z")

</div>

It turns out `@cfunction` behaves much better here.

```julia
julia> const isCalculationValid = Ref(false)
Base.RefValue{Bool}(false)

julia> const inputRef = Ref(1.0)
Base.RefValue{Float64}(1.0)

julia> Base.@cfunction function julia_set_dB_gain(dB_gain::Cdouble)::Cvoid 
           isCalculationValid[]=false # Parameter Set, so invalidate current answer
           inputRef[] = dB_gain
           return nothing
       end Cvoid (Cdouble,)
Ptr{Nothing} @0x00007fbbd0a4cff0

julia> f(x) = ccall(Ptr{Cvoid}(0x00007fbbd0a4cff0), Cvoid, (Cdouble,), x)
f (generic function with 1 method)

julia> @benchmark f(1.0)
BenchmarkTools.Trial: 10000 samples with 1000 evaluations.
 Range (min … max): 5.999 ns … 41.309 ns ┊ GC (min … max): 0.00% … 0.00%
 Time (median): 6.000 ns ┊ GC (median): 0.00%
 Time (mean ± σ): 6.049 ns ± 0.491 ns ┊ GC (mean ± σ): 0.00% ± 0.00%

  █ ▂ ▂ ▁
  █▇█▅▄▄▄▃▃▁▃▃▃▁▁▄▁▁▃▁▁▃▃▃▁▁▁▄▁▃▄▁▁▃▁▁▃▄▁▄▅▆▁▄▁▁▄▁▄▄▃▁▄▃▁▁▃█ █
  6 ns Histogram: log(frequency) by time 7.11 ns <

 Memory estimate: 0 bytes, allocs estimate: 0.

```

So instead of using `@ccallable` you likely want to lookup the function pointer using `cfunction` at the beginning of your program and then use those as callable pointers.

HT @jameson

---

<div class="post-metadata">

**Author:** ![Allan\_Baker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/allan_baker/32/42645_2.png) [@Allan\_Baker](https://discourse.julialang.org/u/Allan_Baker)\
**Post date:** [October 26, 2023, 9:05pm UTC](https://discourse.julialang.org/t/gc-occurs-at-the-worst-time-in-tight-loop-garbage-collection/104934/84 "2023-10-26T21:05:20Z")

</div>

That is really cool how you did that. I was trying to figure out how to do that. We need to just star this as Julia gold.

So how do I make it not heap allocate? Do I need to make a structure for everything and pass by reference?

Is it a bug in Julia?

---

<div class="post-metadata">

**Author:** ![vchuravy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/vchuravy/32/8_2.png) [@vchuravy](https://discourse.julialang.org/u/vchuravy)\
**Post date:** [October 26, 2023, 9:08pm UTC](https://discourse.julialang.org/t/gc-occurs-at-the-worst-time-in-tight-loop-garbage-collection/104934/85 "2023-10-26T21:08:07Z")

</div>

> [@Allan\_Baker](#):
>
> Is it a bug in Julia?

It’s a sub-optimality, partly since the whole create a shared library from Julia story is quite rough…

I think for now the `@cfunction` hack should (while annoying) get you to zero allocations.

I did open up an issue [`@ccallable` allocates for it's arguments · Issue #51894 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/issues/51894)

---

<div class="post-metadata">

**Author:** ![Allan\_Baker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/allan_baker/32/42645_2.png) [@Allan\_Baker](https://discourse.julialang.org/u/Allan_Baker)\
**Post date:** [October 26, 2023, 9:39pm UTC](https://discourse.julialang.org/t/gc-occurs-at-the-worst-time-in-tight-loop-garbage-collection/104934/86 "2023-10-26T21:39:09Z")

</div>

How do I get at the cfunction call from my C function since I’m dealing with a compiled package? I was trying to not require a Julia install. Just the shared libraries. Do I need to shadow it in the shared package library somehow?

---

<div class="post-metadata">

**Author:** ![vchuravy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/vchuravy/32/8_2.png) [@vchuravy](https://discourse.julialang.org/u/vchuravy)\
**Post date:** [October 26, 2023, 11:47pm UTC](https://discourse.julialang.org/t/gc-occurs-at-the-worst-time-in-tight-loop-garbage-collection/104934/87 "2023-10-26T23:47:53Z")

</div>

Are you deleting `libjulia-compiler`? If not you have everything available.

You would do somthing like:

```julia
typedef void(*set_dB_gain)(double);

void main
    // ... init Julia etc

    jl_value_t *jl_ptr = nullptr;
    JL_GC_PUSH1(&jl_ptr);
    jl_ptr = jl_eval_string("Base.@cfunction(julia_set_dB_gain, Cvoid, (Cdouble,))");
    set_dB_gain func = (set_dB_gain) jl_unbox_long(jl_ptr);
    JL_GC_POP();
    

   for ... {
        func(1.0);
    }

```

---

<div class="post-metadata">

**Author:** ![Allan\_Baker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/allan_baker/32/42645_2.png) [@Allan\_Baker](https://discourse.julialang.org/u/Allan_Baker)\
**Post date:** [October 28, 2023, 4:03am UTC](https://discourse.julialang.org/t/gc-occurs-at-the-worst-time-in-tight-loop-garbage-collection/104934/88 "2023-10-28T04:03:38Z")

</div>

> [@vchuravy](#):
>
> `jl_unbox_long`

This is good. I found it in the manual as well.  
It seems with this way, I need a full julia install. Or do I just need to scavenge the julia.h file for the few functions I call and put them in my own header?

---

<div class="post-metadata">

**Author:** ![vchuravy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/vchuravy/32/8_2.png) [@vchuravy](https://discourse.julialang.org/u/vchuravy)\
**Post date:** [October 28, 2023, 12:15pm UTC](https://discourse.julialang.org/t/gc-occurs-at-the-worst-time-in-tight-loop-garbage-collection/104934/89 "2023-10-28T12:15:09Z")

</div>

You don’t need a full Julia install, but you do need `libjulia`, `libjulia-internal`, and `libjulia-codegen` + `julia.h|

I don’t understand why you don’t have that header, but maybe that’s a question for PackageCompiler…?

---

<div class="post-metadata">

**Author:** ![Allan\_Baker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/allan_baker/32/42645_2.png) [@Allan\_Baker](https://discourse.julialang.org/u/Allan_Baker)\
**Post date:** [October 30, 2023, 3:55pm UTC](https://discourse.julialang.org/t/gc-occurs-at-the-worst-time-in-tight-loop-garbage-collection/104934/90 "2023-10-30T15:55:44Z")

</div>

I started looking at all of the includes inside julia.h and it looks like a lot of additional stuff. My thought was, would it be easier to pass pointers as inputs with the @ccallable interface instead of floats for example? That might be easier than hacking the julia.h include.

---

<div class="post-metadata">

**Author:** ![vchuravy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/vchuravy/32/8_2.png) [@vchuravy](https://discourse.julialang.org/u/vchuravy)\
**Post date:** [October 30, 2023, 6:34pm UTC](https://discourse.julialang.org/t/gc-occurs-at-the-worst-time-in-tight-loop-garbage-collection/104934/91 "2023-10-30T18:34:34Z")

</div>

I don’t understand why you need to hack the `julia.h` file?  
You should just include it and don’t change it?

If PackageCompiler doesn’t easily provide it, we should open an issue to change it.

---

<div class="post-metadata">

**Author:** ![Allan\_Baker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/allan_baker/32/42645_2.png) [@Allan\_Baker](https://discourse.julialang.org/u/Allan_Baker)\
**Post date:** [October 30, 2023, 7:46pm UTC](https://discourse.julialang.org/t/gc-occurs-at-the-worst-time-in-tight-loop-garbage-collection/104934/92 "2023-10-30T19:46:37Z")

</div>

> <https://github.com/JuliaLang/PackageCompiler.jl/issues/884>
>
> A bug was found in julia where @ccallable functions allocate memory for each arg…ument triggering garbage collection. For a high-throughput loop this is a no go. It was observed that @cfunction function pointers do not allocate per each argument.
> 
> https://github.com/JuliaLang/julia/issues/51894
> 
> Unfortunately for the shared-library use case with the PackageCompiler.jl using @cfunction interface requires a julia install because the "julia.h" include file is not available to shared libraries created with the PackageCompiler.jl.  
> 
> Can julia.h be made available to shared libraries made with PackageCompiler.jl so that additional embedded functionality can also be accessed via embedded C julia commands? This would save an installation of Julia being required to link in with the shared library.

---

<div class="post-metadata">

**Author:** ![Allan\_Baker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/allan_baker/32/42645_2.png) [@Allan\_Baker](https://discourse.julialang.org/u/Allan_Baker)\
**Post date:** [November 7, 2023, 4:28pm UTC](https://discourse.julialang.org/t/gc-occurs-at-the-worst-time-in-tight-loop-garbage-collection/104934/93 "2023-11-07T16:28:27Z")

</div>

To make complete for the example. I got a segmentation fault for the lines above because the jl\_ptr always came back null. This was because I neglected to include another `jl_eval_string("using myPackage")` where myPackage contains the julia\_set\_dB\_gain function.

---

<div class="post-metadata">

**Author:** ![Allan\_Baker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/allan_baker/32/42645_2.png) [@Allan\_Baker](https://discourse.julialang.org/u/Allan_Baker)\
**Post date:** [November 7, 2023, 4:36pm UTC](https://discourse.julialang.org/t/gc-occurs-at-the-worst-time-in-tight-loop-garbage-collection/104934/94 "2023-11-07T16:36:34Z")

</div>

Also, to debug the segmentation fault, open up a julia session and test all of the `jl_eval_string` lines for mistakes and typos.

[Previous page](https://discourse.julialang.org/t/gc-occurs-at-the-worst-time-in-tight-loop-garbage-collection/104934.md?page=4)