# Why is garbage collection called here?

**URL:** <https://discourse.julialang.org/t/why-is-garbage-collection-called-here/109870>\
**Category:** Performance\
**Created:** [February 7, 2024, 1:16pm UTC](https://discourse.julialang.org/t/why-is-garbage-collection-called-here/109870 "2024-02-07T13:16:56Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ahmed\_Salih](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ahmed_salih/32/206579_2.png) [@Ahmed\_Salih](https://discourse.julialang.org/u/Ahmed_Salih)\
**Post date:** [February 7, 2024, 1:16pm UTC](https://discourse.julialang.org/t/why-is-garbage-collection-called-here/109870/1 "2024-02-07T13:16:56Z")

</div>

Hello!

I’ve been hitting my head on the wall for a few days now. I have a function which when using `@profview` seems to call the garbage collection (red bar), but it does not allocate. It is `updatexij!` (which is resized every iteration in `ResizeBuffers!`) in the following screenshot:

 ![image](https://global.discourse-cdn.com/julialang/original/3X/f/b/fb09481f2092ad907e39cdafee92a18c77d23fb3.png)

When I look into the function it self:

 ![image](https://global.discourse-cdn.com/julialang/original/3X/f/0/f011a1f5d448fba464f184c3dbc867eb86bd2d4c.png)

No red bars are present.

How do I debug this and figure out where the garbage collection is coming from?

Kind regards

---

<div class="post-metadata">

**Author:** ![abraemer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/abraemer/32/51403_2.png) [@abraemer](https://discourse.julialang.org/u/abraemer)\
**Post date:** [February 7, 2024, 1:34pm UTC](https://discourse.julialang.org/t/why-is-garbage-collection-called-here/109870/2 "2024-02-07T13:34:54Z")

</div>

I don’t think that garbage collection is generally attributed to the lines that did the allocation because GC runs at essentially random times.

Other points to check:

- is everything type stable at the call site? Dynamic dispatch causes allocations
- you have wrapped things in `@timeit` that might cause allocation maybe? Do you still see the GC runs if you remove the timing macros?
- are the GC timings/valuea the same between runs?
- GC generally runs when there are allocations. So to reduce GC you need to reduce allocations. You can also track allocations. See here  
[Profiling · The Julia Language](https://docs.julialang.org/en/v1/manual/profile/#Line-by-Line-Allocation-Tracking)

---

<div class="post-metadata">

**Author:** ![Ahmed\_Salih](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ahmed_salih/32/206579_2.png) [@Ahmed\_Salih](https://discourse.julialang.org/u/Ahmed_Salih)\
**Post date:** [February 7, 2024, 2:30pm UTC](https://discourse.julialang.org/t/why-is-garbage-collection-called-here/109870/3 "2024-02-07T14:30:57Z")

</div>

Okay got it, it is just because these are pretty much the only places I see GC in my code.

- is everything type stable at the call site? Dynamic dispatch causes allocations

I believe so. Using JET.

```julia
julia> @report_opt updatexᵢⱼ!(xᵢⱼ,system.nb.list,Position)
No errors detected

julia> @report_call updatexᵢⱼ!(xᵢⱼ,system.nb.list,Position)
No errors detected

```

Using `@code_warntype`:

```julia
@code_warntype updatexᵢⱼ!(xᵢⱼ,system.nb.list,Position)
MethodInstance for SPHExample.SimulationEquations.updatexᵢⱼ!(::Vector{SVector{3, Float64}}, ::Vector{Tuple{Int64, Int64, Float64}}, ::Vector{SVector{3, Float64}})
  from updatexᵢⱼ!(xᵢⱼ, list, points) @ SPHExample.SimulationEquations c:\git\SPHExample\src\SimulationEquations.jl:220
Arguments
  #self#::Core.Const(SPHExample.SimulationEquations.updatexᵢⱼ!)
  xᵢⱼ::Vector{SVector{3, Float64}}
  list::Vector{Tuple{Int64, Int64, Float64}}
  points::Vector{SVector{3, Float64}}
Locals
  @_5::Union{Nothing, Tuple{Tuple{Int64, Tuple{Int64, Int64, Float64}}, Tuple{Int64, Int64}}}
  @_6::Int64
  L::Tuple{Int64, Int64, Float64}
  iter::Int64
  j::Int64
  i::Int64
Body::Nothing
1 ─ Core.NewvarNode(:(@_5))
│ %2 = SPHExample.SimulationEquations.length(xᵢⱼ)::Int64
│ %3 = SPHExample.SimulationEquations.length(list)::Int64
│ %4 = (%2 != %3)::Bool
└── goto #3 if not %4
2 ─ %6 = SPHExample.SimulationEquations.length(list)::Int64
└── SPHExample.SimulationEquations.resize!(xᵢⱼ, %6)
3 ┄ %8 = SPHExample.SimulationEquations.enumerate(list)::Base.Iterators.Enumerate{Vector{Tuple{Int64, Int64, Float64}}}
│ (@_5 = Base.iterate(%8))
│ %10 = (@_5 === nothing)::Bool
│ %11 = Base.not_int(%10)::Bool
└── goto #6 if not %11
4 ┄ %13 = @_5::Tuple{Tuple{Int64, Tuple{Int64, Int64, Float64}}, Tuple{Int64, Int64}}
│ %14 = Core.getfield(%13, 1)::Tuple{Int64, Tuple{Int64, Int64, Float64}}
│ %15 = Base.indexed_iterate(%14, 1)::Core.PartialStruct(Tuple{Int64, Int64}, Any[Int64, Core.Const(2)])
│ (iter = Core.getfield(%15, 1))
│ (@_6 = Core.getfield(%15, 2))
│ %18 = Base.indexed_iterate(%14, 2, @_6::Core.Const(2))::Core.PartialStruct(Tuple{Tuple{Int64, Int64, Float64}, Int64}, Any[Tuple{Int64, Int64, Float64}, Core.Const(3)])
│ (L = Core.getfield(%18, 1))
│ %20 = Core.getfield(%13, 2)::Tuple{Int64, Int64}
│ (i = Base.getindex(L, 1))
│ (j = Base.getindex(L, 2))
│ %23 = Base.getindex(points, i)::SVector{3, Float64}
│ %24 = Base.getindex(points, j)::SVector{3, Float64}
│ %25 = (%23 - %24)::SVector{3, Float64}
│ Base.setindex!(xᵢⱼ, %25, iter)
│ (@_5 = Base.iterate(%8, %20))
│ %28 = (@_5 === nothing)::Bool
│ %29 = Base.not_int(%28)::Bool
└── goto #6 if not %29
5 ─ goto #4
6 ┄ return nothing

```

- you have wrapped things in `@timeit` that might cause allocation maybe? Do you still see the GC runs if you remove the timing macros?

Commented out @timeit, still see GC:

 ![image](https://global.discourse-cdn.com/julialang/original/3X/1/d/1d9e0c1800e0d526ae4825dc172ba2929bd6f1d9.png)

- are the GC timings/valuea the same between runs?

Yes, similar.

- GC generally runs when there are allocations. So to reduce GC you need to reduce allocations. You can also track allocations. See here

I’ve done `@benchmark` on the function and it has zero allocations. This is why I struggle to find out the root cause.

I would really like to fix this for max performance, so any help appreciated 🙂

---

<div class="post-metadata">

**Author:** ![abraemer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/abraemer/32/51403_2.png) [@abraemer](https://discourse.julialang.org/u/abraemer)\
**Post date:** [February 7, 2024, 2:38pm UTC](https://discourse.julialang.org/t/why-is-garbage-collection-called-here/109870/4 "2024-02-07T14:38:37Z")

</div>

> [@Ahmed\_Salih](#):
>
> I believe so. Using JET.
> 
> ```julia
> julia> @report_opt updatexᵢⱼ!(xᵢⱼ,system.nb.list,Position)
> No errors detected
> julia> @report_call updatexᵢⱼ!(xᵢⱼ,system.nb.list,Position)
> No errors detected
> 
> ```

That is of course also good information but not what I meant. Sorry for being unclear. Dynamic dispatch would occur when you call `updatexᵢⱼ` if the function surrounding the call has type instabilities. So suppose you have the code snippets you showed inside some larger function:

```julia
function large_function(...)
    #...
    updatexᵢⱼ!(xᵢⱼ,system.nb.list,Position)
    # ... more code
end

```

You showed that `updatexᵢⱼ!` is good (does not allocate and is type stable). So now we need to check the `large_function` for allocations/type instabilities 🙂

---

<div class="post-metadata">

**Author:** ![Ahmed\_Salih](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ahmed_salih/32/206579_2.png) [@Ahmed\_Salih](https://discourse.julialang.org/u/Ahmed_Salih)\
**Post date:** [February 7, 2024, 3:02pm UTC](https://discourse.julialang.org/t/why-is-garbage-collection-called-here/109870/5 "2024-02-07T15:02:14Z")

</div>

Got it, thanks!

I now do it on the `large_function` in my case called `RunSimulation`,

`@report_opt target_modules=(@ __MODULE__ ,)`:

```julia
═════ 18 possible errors found ═════
┌ kwcall(::@NamedTuple{FluidCSV::String, BoundCSV::String, SimulationMetaData::SimulationMetaData, SimulationConstants::SimulationConstants{Float64}}, ::typeof(RunSimulation)) @ Main c:\git\SPHExample\example\MainSimulation.jl:58
│┌ RunSimulation(; FluidCSV::String, BoundCSV::String, SimulationMetaData::SimulationMetaData, SimulationConstants::SimulationConstants{Float64}) @ Main c:\git\SPHExample\example\MainSimulation.jl:74 
││ runtime dispatch detected: Core.kwcall(%16::@NamedTuple{NumberOfParticles::Int64}, %14::Type{SimulationDataResults{3, _A}} where _A)::SimulationDataResults{3}
│└────────────────────
│┌ RunSimulation(; FluidCSV::String, BoundCSV::String, SimulationMetaData::SimulationMetaData, SimulationConstants::SimulationConstants{Float64}) @ Main c:\git\SPHExample\example\MainSimulation.jl:78 
││ runtime dispatch detected: vcat(%53::AbstractVector, %54::AbstractVector)::Any
│└────────────────────
│┌ RunSimulation(; FluidCSV::String, BoundCSV::String, SimulationMetaData::SimulationMetaData, SimulationConstants::SimulationConstants{Float64}) @ Main c:\git\SPHExample\example\MainSimulation.jl:78 
││ runtime dispatch detected: Array(%55::Any)::Any
│└────────────────────
│┌ RunSimulation(; FluidCSV::String, BoundCSV::String, SimulationMetaData::SimulationMetaData, SimulationConstants::SimulationConstants{Float64}) @ Main c:\git\SPHExample\example\MainSimulation.jl:78 
││ runtime dispatch detected: Base.broadcasted(identity, %56::Any)::Any
│└────────────────────
│┌ RunSimulation(; FluidCSV::String, BoundCSV::String, SimulationMetaData::SimulationMetaData, SimulationConstants::SimulationConstants{Float64}) @ Main c:\git\SPHExample\example\MainSimulation.jl:78 
││ runtime dispatch detected: Base.materialize!(%20::Vector, %57::Any)::Any
│└────────────────────
│┌ RunSimulation(; FluidCSV::String, BoundCSV::String, SimulationMetaData::SimulationMetaData, SimulationConstants::SimulationConstants{Float64}) @ Main c:\git\SPHExample\example\MainSimulation.jl:95 
││ runtime dispatch detected: create_vtp_file(SimulationMetaData::SimulationMetaData, SimulationConstants::SimulationConstants{Float64}, %17::SimulationDataResults{3})::Any
│└────────────────────
│┌ RunSimulation(; FluidCSV::String, BoundCSV::String, SimulationMetaData::SimulationMetaData, SimulationConstants::SimulationConstants{Float64}) @ Main c:\git\SPHExample\example\MainSimulation.jl:102││ runtime dispatch detected: zeros(%3::DataType, %718::Tuple{Int64})::Vector
│└────────────────────
│┌ RunSimulation(; FluidCSV::String, BoundCSV::String, SimulationMetaData::SimulationMetaData, SimulationConstants::SimulationConstants{Float64}) @ Main c:\git\SPHExample\example\MainSimulation.jl:103││ runtime dispatch detected: zeros(%734::Union{Type{Any}, Type{E} where E<:(SVector{3})}, %720::Tuple{Int64})::Vector{T} where T<:(SVector{3})
│└────────────────────
│┌ RunSimulation(; FluidCSV::String, BoundCSV::String, SimulationMetaData::SimulationMetaData, SimulationConstants::SimulationConstants{Float64}) @ Main c:\git\SPHExample\example\MainSimulation.jl:105││ runtime dispatch detected: zeros(%3::DataType, %718::Tuple{Int64})::Vector
│└────────────────────
│┌ RunSimulation(; FluidCSV::String, BoundCSV::String, SimulationMetaData::SimulationMetaData, SimulationConstants::SimulationConstants{Float64}) @ Main c:\git\SPHExample\example\MainSimulation.jl:106││ runtime dispatch detected: zeros(%734::Union{Type{Any}, Type{E} where E<:(SVector{3})}, %720::Tuple{Int64})::Vector{T} where T<:(SVector{3})
│└────────────────────
│┌ RunSimulation(; FluidCSV::String, BoundCSV::String, SimulationMetaData::SimulationMetaData, SimulationConstants::SimulationConstants{Float64}) @ Main c:\git\SPHExample\example\MainSimulation.jl:107││ runtime dispatch detected: zeros(%734::Union{Type{Any}, Type{E} where E<:(SVector{3})}, %720::Tuple{Int64})::Vector{T} where T<:(SVector{3})
│└────────────────────
│┌ RunSimulation(; FluidCSV::String, BoundCSV::String, SimulationMetaData::SimulationMetaData, SimulationConstants::SimulationConstants{Float64}) @ Main c:\git\SPHExample\example\MainSimulation.jl:109││ runtime dispatch detected: zeros(%3::DataType, %718::Tuple{Int64})::Vector
│└────────────────────
│┌ RunSimulation(; FluidCSV::String, BoundCSV::String, SimulationMetaData::SimulationMetaData, SimulationConstants::SimulationConstants{Float64}) @ Main c:\git\SPHExample\example\MainSimulation.jl:112││ runtime dispatch detected: zeros(%734::Union{Type{Any}, Type{E} where E<:(SVector{3})}, %720::Tuple{Int64})::Vector{T} where T<:(SVector{3})
│└────────────────────
│┌ RunSimulation(; FluidCSV::String, BoundCSV::String, SimulationMetaData::SimulationMetaData, SimulationConstants::SimulationConstants{Float64}) @ Main c:\git\SPHExample\example\MainSimulation.jl:113││ runtime dispatch detected: zeros(%734::Union{Type{Any}, Type{E} where E<:(SVector{3})}, %720::Tuple{Int64})::Vector{T} where T<:(SVector{3})
│└────────────────────
│┌ RunSimulation(; FluidCSV::String, BoundCSV::String, SimulationMetaData::SimulationMetaData, SimulationConstants::SimulationConstants{Float64}) @ Main c:\git\SPHExample\example\MainSimulation.jl:114││ runtime dispatch detected: zeros(%3::DataType, %718::Tuple{Int64})::Vector
│└────────────────────
│┌ RunSimulation(; FluidCSV::String, BoundCSV::String, SimulationMetaData::SimulationMetaData, SimulationConstants::SimulationConstants{Float64}) @ Main c:\git\SPHExample\example\MainSimulation.jl:115││ runtime dispatch detected: zeros(%3::DataType, %718::Tuple{Int64})::Vector
│└────────────────────
│┌ RunSimulation(; FluidCSV::String, BoundCSV::String, SimulationMetaData::SimulationMetaData, SimulationConstants::SimulationConstants{Float64}) @ Main c:\git\SPHExample\example\MainSimulation.jl:117││ runtime dispatch detected: zeros(%3::DataType, %718::Tuple{Int64})::Vector
│└────────────────────
│┌ RunSimulation(; FluidCSV::String, BoundCSV::String, SimulationMetaData::SimulationMetaData, SimulationConstants::SimulationConstants{Float64}) @ Main c:\git\SPHExample\example\MainSimulation.jl:120││ runtime dispatch detected: Core.kwcall(%791::NamedTuple{(:x, :cutoff, :parallel), <:Tuple{Array{SVector{3, _A}, 1} where _A, Float64, Bool}}, InPlaceNeighborList)::InPlaceNeighborList{B, _A, _B, CellListMap.NeighborList{Float64}} where {B<:(Box{NonPeriodicCell}), _A, _B}
│└────────────────────

```

`@report_call target_modules=(@ __MODULE__ ,)`:  
No errors detected

`@code_warntype`:

```julia
MethodInstance for Core.kwcall(::@NamedTuple{FluidCSV::String, BoundCSV::String, SimulationMetaData::SimulationMetaData, SimulationConstants::SimulationConstants{Float64}}, ::typeof(RunSimulation))
  from kwcall(::NamedTuple, ::typeof(RunSimulation)) @ Main c:\git\SPHExample\example\MainSimulation.jl:58
Arguments
  _::Core.Const(Core.kwcall)
  @_2::@NamedTuple{FluidCSV::String, BoundCSV::String, SimulationMetaData::SimulationMetaData, SimulationConstants::SimulationConstants{Float64}}
  @_3::Core.Const(RunSimulation)
Locals
  @_4::Union{SimulationConstants{Float64}, String, SimulationMetaData}
  FluidCSV::String
  BoundCSV::String
  SimulationMetaData::SimulationMetaData
  SimulationConstants::SimulationConstants{Float64}
Body::Bool
1 ── Core.NewvarNode(:(@_4))
│ %2 = Core.isdefined(@_2, :FluidCSV)::Core.Const(true)
└─── goto #6 if not %2
2 ── %4 = Core.getfield(@_2, :FluidCSV)::String
│ %5 = (%4 isa Main.String)::Core.Const(true)
└─── goto #4 if not %5
3 ── goto #5
4 ── Core.Const(:(%new(Core.TypeError, Symbol("keyword argument"), :FluidCSV, Main.String, %4)))
└─── Core.Const(:(Core.throw(%8)))
5 ┄─ (@_4 = %4)
└─── goto #7
6 ── Core.Const(:(Core.UndefKeywordError(:FluidCSV)))
└─── Core.Const(:(@_4 = Core.throw(%12)))
7 ┄─ %14 = @_4::String
│ (FluidCSV = %14)
│ %16 = Core.isdefined(@_2, :BoundCSV)::Core.Const(true)
└─── goto #12 if not %16
8 ── %18 = Core.getfield(@_2, :BoundCSV)::String
│ %19 = (%18 isa Main.String)::Core.Const(true)
└─── goto #10 if not %19
9 ── goto #11
10 ─ Core.Const(:(%new(Core.TypeError, Symbol("keyword argument"), :BoundCSV, Main.String, %18)))
└─── Core.Const(:(Core.throw(%22)))
11 ┄ (@_4 = %18)
└─── goto #13
12 ─ Core.Const(:(Core.UndefKeywordError(:BoundCSV)))
└─── Core.Const(:(@_4 = Core.throw(%26)))
13 ┄ %28 = @_4::String
│ (BoundCSV = %28)
│ %30 = Core.isdefined(@_2, :SimulationMetaData)::Core.Const(true)
└─── goto #18 if not %30
14 ─ %32 = Core.getfield(@_2, :SimulationMetaData)::SimulationMetaData
│ %33 = (%32 isa Main.SimulationMetaData)::Core.Const(true)
└─── goto #16 if not %33
15 ─ goto #17
16 ─ Core.Const(:(%new(Core.TypeError, Symbol("keyword argument"), :SimulationMetaData, Main.SimulationMetaData, %32)))
└─── Core.Const(:(Core.throw(%36)))
17 ┄ (@_4 = %32)
└─── goto #19
18 ─ Core.Const(:(Core.UndefKeywordError(:SimulationMetaData)))
└─── Core.Const(:(@_4 = Core.throw(%40)))
19 ┄ %42 = @_4::SimulationMetaData
│ (SimulationMetaData = %42)
│ %44 = Core.isdefined(@_2, :SimulationConstants)::Core.Const(true)
└─── goto #24 if not %44
20 ─ %46 = Core.getfield(@_2, :SimulationConstants)::SimulationConstants{Float64}
│ %47 = (%46 isa Main.SimulationConstants)::Core.Const(true)
└─── goto #22 if not %47
21 ─ goto #23
22 ─ Core.Const(:(%new(Core.TypeError, Symbol("keyword argument"), :SimulationConstants, Main.SimulationConstants, %46)))
└─── Core.Const(:(Core.throw(%50)))
23 ┄ (@_4 = %46)
└─── goto #25
24 ─ Core.Const(:(Core.UndefKeywordError(:SimulationConstants)))
└─── Core.Const(:(@_4 = Core.throw(%54)))
25 ┄ %56 = @_4::SimulationConstants{Float64}
│ (SimulationConstants = %56)
│ %58 = (:FluidCSV, :BoundCSV, :SimulationMetaData, :SimulationConstants)::Core.Const((:FluidCSV, :BoundCSV, :SimulationMetaData, :SimulationConstants))
│ %59 = Core.apply_type(Core.NamedTuple, %58)::Core.Const(NamedTuple{(:FluidCSV, :BoundCSV, :SimulationMetaData, :SimulationConstants)})
│ %60 = Base.structdiff(@_2, %59)::Core.Const(NamedTuple())
│ %61 = Base.pairs(%60)::Core.Const(Base.Pairs{Symbol, Union{}, Tuple{}, @NamedTuple{}}())
│ %62 = Base.isempty(%61)::Core.Const(true)
└─── goto #27 if not %62
26 ─ goto #28
27 ─ Core.Const(:(Base.kwerr(@_2, @_3)))
28 ┄ %66 = Main.:(var"#RunSimulation#25")(FluidCSV, BoundCSV, SimulationMetaData, SimulationConstants, @_3)::Core.Const(false)
└─── return %66

```

Is this more a long the lines what you asked for? 🙂

---

<div class="post-metadata">

**Author:** ![abraemer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/abraemer/32/51403_2.png) [@abraemer](https://discourse.julialang.org/u/abraemer)\
**Post date:** [February 7, 2024, 3:14pm UTC](https://discourse.julialang.org/t/why-is-garbage-collection-called-here/109870/6 "2024-02-07T15:14:12Z")

</div>

Yes it is 🙂  
And we see that there is indeed some type-instability here. Not sure whether it is relevant just yet. The `@code_warntype` actually just shows the types inside a “wrapper” that Julia generated to handle keyword and default arguments. I am unsure how the Jet.jl output needs to be interpreted, i.e. whether that means there truly is type instability inside `RunSimulation` or just in the “wrapper” function. So we need to get the `@code_warntype` of the inner function. You can do this with [`Cthulhu.jl`](https://github.com/JuliaDebug/Cthulhu.jl). This allows you to recursively descend into the function calls and print the `code_warntype`. So instead of `@code_warntype RunSimulation(...)` you use `@descend RunSimulation(...)` and then you can interactively descend into the call.  
Disclaimer: I have so far not done this myself.

Can I see the full code of `RunSimulation` somewhere or can you post it? That would help me to get a feeling for what’s happening 🙂

---

<div class="post-metadata">

**Author:** ![Ahmed\_Salih](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ahmed_salih/32/206579_2.png) [@Ahmed\_Salih](https://discourse.julialang.org/u/Ahmed_Salih)\
**Post date:** [February 7, 2024, 3:21pm UTC](https://discourse.julialang.org/t/why-is-garbage-collection-called-here/109870/7 "2024-02-07T15:21:29Z")

</div>

Yes!

The code for `RunSimulation` is here: [SPHExample/example/MainSimulation.jl at optimize-code-execution · AhmedSalih3d/SPHExample · GitHub](https://github.com/AhmedSalih3d/SPHExample/blob/optimize-code-execution/example/MainSimulation.jl)

The first part of the function loads in some stuff and allocates once, then the main loop inside of the function calculates different properties.

Kind regards

---

<div class="post-metadata">

**Author:** ![abraemer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/abraemer/32/51403_2.png) [@abraemer](https://discourse.julialang.org/u/abraemer)\
**Post date:** [February 7, 2024, 3:56pm UTC](https://discourse.julialang.org/t/why-is-garbage-collection-called-here/109870/8 "2024-02-07T15:56:45Z")

</div>

Ok that makes everything clearer. `RunSimulation` is infact quite type unstable 🙂  
See you pass in `SimulationMetaData` and that contains abstractly typed fields (`CurrentTimeStep::AbstractFloat`, `TotalTime::AbstractFloat`) and even types in variables (`FloatType::DataType = Float64`, `IntType::DataType = Int64`) that Julia cannot know at compile time of `RunSimulation`. I didn’t check the other arguments to `RunSimulation`.

There are a couple of ways out from here:

- You could move all relevant type information actually into the types of the arguments to `RunSimulation`, i.e. make `SimulationMetaData` a parametric type like so:

```julia
@with_kw mutable struct SimulationMetaData{F, FloatType, IntType} where {FloatType <: AbstractFloat, IntType <: Integer}
    SimulationName::String
    SaveLocation::String
    HourGlass::TimerOutput = TimerOutput()
    Iteration::Int = 0
    MaxIterations::Int = 1000
    OutputIteration::Int = 50
    CurrentTimeStep::F = 0.0
    TotalTime::F = 0.0
    SilentOutput::Bool = false
    ThreadsCPU::Int = Threads.nthreads()
    ProgressSpecification::Progress = Progress(MaxIterations)       
end

```

and do the same for the other parameters.

- Another idea would be to use a [function barrier](https://docs.julialang.org/en/v1/manual/performance-tips/#kernel-functions). That means you restructure your code a bit and preallocate all things inside `RunSimulation` and then move the actual computation to a “kernel function” (called e.g. “RunSimulation\_kernel”) which is called by `RunSimulation` with a ridiculous amount of arguments 😃 Like so

```julia
function RunSimulation(;FluidCSV::String,
                        BoundCSV::String,
                        SimulationMetaData::SimulationMetaData,
                        SimulationConstants::SimulationConstants
)
    @unpack ...
    # do all the preallocation stuff
    # ....
    Pressureᵢ = zeros(FloatType, SizeOfParticlesI1)

    # Initialize the system system.nb.list
    system = InPlaceNeighborList(x=Position, cutoff=2*h, parallel=true)
    
    # now call the kernel function to perform the actual computation
    # this causes a single dynamic dispatch and after that everything is type stable
    RunSimulation_kernel(system, Pressureᵢ, ... ,... )

    # now the rest of RunSimulation
    show(HourGlass,sortby=:name)
    show(HourGlass)
    disable_timer!(HourGlass)
end

function RunSimulation_kernel(....)
    @inbounds for SimulationMetaData.Iteration = 1:MaxIterations
    # Be sure to update and retrieve the updated neighbour system.nb.list at each time step
   # ... rest of the loops and computation
end

```

The latter has the great advantage that you don’t need to change a lot of code to make the computation fast. Although the setup/preallocation will stay type unstable but that doesn’t really matter as long as the computation takes the majority of the runtime anyways.

---

<div class="post-metadata">

**Author:** ![Ahmed\_Salih](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ahmed_salih/32/206579_2.png) [@Ahmed\_Salih](https://discourse.julialang.org/u/Ahmed_Salih)\
**Post date:** [February 8, 2024, 9:08am UTC](https://discourse.julialang.org/t/why-is-garbage-collection-called-here/109870/9 "2024-02-08T09:08:56Z")

</div>

What I don’t like about the second suggestion is that it has to have these 100s of arguments… so I went ahead and gave your first suggestion a shot, since that was quite easy to do. Alhamdu lillah I did that! I paramterized `SimulationMetaData` similar to what you suggested:

```julia
@with_kw mutable struct SimulationMetaData{FloatType <: AbstractFloat}
    SimulationName::String
    SaveLocation::String
    HourGlass::TimerOutput = TimerOutput()
    Iteration::Int = 0
    MaxIterations::Int = 1000
    OutputIteration::Int = 50
    CurrentTimeStep::FloatType = 0
    TotalTime::FloatType = 0
    SilentOutput::Bool = false
    ThreadsCPU::Int = Threads.nthreads()
    ProgressSpecification::Progress = Progress(MaxIterations)       
end

```

**And understood what you meant about the compiler not knowing the datatype when it is a field instead of a type parameter**

Then I went ahead and did the checks:

`@report_opt target_modules=(@ __MODULE__ ,)`

 ![image](https://global.discourse-cdn.com/julialang/original/3X/c/0/c05add60f475fb627aaecaeb48091b6111598104.png)

Which is basically this line:

`Density .= Array([DF_FLUID.Rhop;DF_BOUND.Rhop])`

Any suggestion on how to fix?

`@report_call target_modules=(@ __MODULE__ ,)`  
No errors detected

`@code_warntype`

```julia
MethodInstance for Core.kwcall(::@NamedTuple{FluidCSV::String, BoundCSV::String, SimMetaData::SimulationMetaData{Float64}, SimConstants::SimulationConstants{Float64}}, ::typeof(RunSimulation))
  from kwcall(::NamedTuple, ::typeof(RunSimulation)) @ Main c:\git\SPHExample\example\MainSimulation.jl:58
Arguments
  _::Core.Const(Core.kwcall)
  @_2::@NamedTuple{FluidCSV::String, BoundCSV::String, SimMetaData::SimulationMetaData{Float64}, SimConstants::SimulationConstants{Float64}}
  @_3::Core.Const(RunSimulation)
Locals
  @_4::Union{SimulationConstants{Float64}, String, SimulationMetaData{Float64}}
  FluidCSV::String
  BoundCSV::String
  SimMetaData::SimulationMetaData{Float64}
  SimConstants::SimulationConstants{Float64}
Body::Bool
1 ── Core.NewvarNode(:(@_4))
│ %2 = Core.isdefined(@_2, :FluidCSV)::Core.Const(true)
└─── goto #6 if not %2
2 ── %4 = Core.getfield(@_2, :FluidCSV)::String
│ %5 = (%4 isa Main.String)::Core.Const(true)
└─── goto #4 if not %5
3 ── goto #5
4 ── Core.Const(:(%new(Core.TypeError, Symbol("keyword argument"), :FluidCSV, Main.String, %4)))
└─── Core.Const(:(Core.throw(%8)))
5 ┄─ (@_4 = %4)
└─── goto #7
6 ── Core.Const(:(Core.UndefKeywordError(:FluidCSV)))
└─── Core.Const(:(@_4 = Core.throw(%12)))
7 ┄─ %14 = @_4::String
│ (FluidCSV = %14)
│ %16 = Core.isdefined(@_2, :BoundCSV)::Core.Const(true)
└─── goto #12 if not %16
8 ── %18 = Core.getfield(@_2, :BoundCSV)::String
│ %19 = (%18 isa Main.String)::Core.Const(true)
└─── goto #10 if not %19
9 ── goto #11
10 ─ Core.Const(:(%new(Core.TypeError, Symbol("keyword argument"), :BoundCSV, Main.String, %18)))
└─── Core.Const(:(Core.throw(%22)))
11 ┄ (@_4 = %18)
└─── goto #13
12 ─ Core.Const(:(Core.UndefKeywordError(:BoundCSV)))
└─── Core.Const(:(@_4 = Core.throw(%26)))
13 ┄ %28 = @_4::String
│ (BoundCSV = %28)
│ %30 = Core.isdefined(@_2, :SimMetaData)::Core.Const(true)
└─── goto #18 if not %30
14 ─ %32 = Core.getfield(@_2, :SimMetaData)::SimulationMetaData{Float64}
│ %33 = (%32 isa Main.SimulationMetaData)::Core.Const(true)
└─── goto #16 if not %33
15 ─ goto #17
16 ─ Core.Const(:(%new(Core.TypeError, Symbol("keyword argument"), :SimMetaData, Main.SimulationMetaData, %32)))
└─── Core.Const(:(Core.throw(%36)))
17 ┄ (@_4 = %32)
└─── goto #19
18 ─ Core.Const(:(Core.UndefKeywordError(:SimMetaData)))
└─── Core.Const(:(@_4 = Core.throw(%40)))
19 ┄ %42 = @_4::SimulationMetaData{Float64}
│ (SimMetaData = %42)
│ %44 = Core.isdefined(@_2, :SimConstants)::Core.Const(true)
└─── goto #24 if not %44
20 ─ %46 = Core.getfield(@_2, :SimConstants)::SimulationConstants{Float64}
│ %47 = (%46 isa Main.SimulationConstants)::Core.Const(true)
└─── goto #22 if not %47
21 ─ goto #23
22 ─ Core.Const(:(%new(Core.TypeError, Symbol("keyword argument"), :SimConstants, Main.SimulationConstants, %46)))
└─── Core.Const(:(Core.throw(%50)))
23 ┄ (@_4 = %46)
└─── goto #25
24 ─ Core.Const(:(Core.UndefKeywordError(:SimConstants)))
└─── Core.Const(:(@_4 = Core.throw(%54)))
25 ┄ %56 = @_4::SimulationConstants{Float64}
│ (SimConstants = %56)
│ %58 = (:FluidCSV, :BoundCSV, :SimMetaData, :SimConstants)::Core.Const((:FluidCSV, :BoundCSV, :SimMetaData, :SimConstants))
│ %59 = Core.apply_type(Core.NamedTuple, %58)::Core.Const(NamedTuple{(:FluidCSV, :BoundCSV, :SimMetaData, :SimConstants)})
│ %60 = Base.structdiff(@_2, %59)::Core.Const(NamedTuple())
│ %61 = Base.pairs(%60)::Core.Const(Base.Pairs{Symbol, Union{}, Tuple{}, @NamedTuple{}}())
│ %62 = Base.isempty(%61)::Core.Const(true)
└─── goto #27 if not %62
26 ─ goto #28
27 ─ Core.Const(:(Base.kwerr(@_2, @_3)))
28 ┄ %66 = Main.:(var"#RunSimulation#101")(FluidCSV, BoundCSV, SimMetaData, SimConstants, @_3)::Core.Const(false)
└─── return %66

```

And **finally** for the big reveal!

No GC happening!

 ![image](https://global.discourse-cdn.com/julialang/original/3X/1/0/10d410e39f12dee656b96fe2a952381478f1880e.png)

And now look at my `@timeroutput` measurements, all allocations pretty much vanished and the ones left I know why!

```julia
                                                                      Time Allocations
                                                             ─────────────────────── ────────────────────────
                      Tot / % measured: 162s / 91.6% 1.19GiB / 38.4%

 Section ncalls time %tot avg alloc %tot avg
 ──────────────────────────────────────────────────────────────────────────────────────────────────────────────
 2| DDT 10.0k 29.2s 19.7% 2.92ms 27.1MiB 5.8% 2.78KiB
 2| DDT2 10.0k 28.6s 19.3% 2.86ms 26.4MiB 5.6% 2.70KiB
 0 | Update Neighbour system.nb.list 10.0k 22.5s 15.2% 2.25ms 164MiB 35.1% 16.8KiB
 1 | Update xᵢⱼ, kernel values and kernel gradient 10.0k 20.7s 14.0% 2.07ms 0.00B 0.0% 0.00B
 4| OutputVTP 10.0k 11.0s 7.4% 1.10ms 243MiB 52.0% 24.9KiB
 2| ∂Πᵢⱼ∂t! 10.0k 8.97s 6.0% 896μs 0.00B 0.0% 0.00B
 2| ∂Πᵢⱼ∂t!2 10.0k 8.95s 6.0% 895μs 0.00B 0.0% 0.00B
 2| ∂vᵢ∂t! 10.0k 5.17s 3.5% 517μs 0.00B 0.0% 0.00B
 2| ∂vᵢ∂t!2 10.0k 5.17s 3.5% 517μs 0.00B 0.0% 0.00B
 2| updatexᵢⱼ! 10.0k 4.04s 2.7% 404μs 0.00B 0.0% 0.00B
 0 | Reset arrays to zero and resize L arrays 10.0k 699ms 0.5% 69.8μs 7.03MiB 1.5% 737B
 2| Pressure 10.0k 546ms 0.4% 54.6μs 0.00B 0.0% 0.00B
 2| Pressure2 10.0k 544ms 0.4% 54.4μs 0.00B 0.0% 0.00B
 3| Calculating time step 10.0k 520ms 0.4% 52.0μs 0.00B 0.0% 0.00B
 2| Positionₙ⁺ 10.0k 345ms 0.2% 34.5μs 0.00B 0.0% 0.00B
 2| Position 10.0k 306ms 0.2% 30.6μs 0.00B 0.0% 0.00B
 2| vₙ⁺ 10.0k 196ms 0.1% 19.6μs 0.00B 0.0% 0.00B
 2| Velocity 10.0k 183ms 0.1% 18.3μs 0.00B 0.0% 0.00B
 2| Acceleration2 10.0k 168ms 0.1% 16.8μs 0.00B 0.0% 0.00B
 2| Gravity 10.0k 165ms 0.1% 16.5μs 0.00B 0.0% 0.00B
 2| DensityEpsi! 10.0k 108ms 0.1% 10.8μs 0.00B 0.0% 0.00B
 2| ρₙ⁺ 10.0k 87.7ms 0.1% 8.77μs 0.00B 0.0% 0.00B
 2| LimitDensityAtBoundary!(ρₙ⁺) 10.0k 46.4ms 0.0% 4.64μs 0.00B 0.0% 0.00B
 2| LimitDensityAtBoundary!(Density) 10.0k 46.3ms 0.0% 4.63μs 0.00B 0.0% 0.00B
 ──────────────────────────────────────────────────────────────────────────────────────────────────────────────

```

@abraemer from the bottom of my heart, thank you so much for taking the time to look through the code. I have been chasing this GC and minor allocations previously for quite a few days now and such a simple change to the `SimulationMetaData` struct taught me so much and fixed the issue more or less completely.

Thanks a ton!

Kind regards

---

<div class="post-metadata">

**Author:** ![abraemer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/abraemer/32/51403_2.png) [@abraemer](https://discourse.julialang.org/u/abraemer)\
**Post date:** [February 8, 2024, 10:17am UTC](https://discourse.julialang.org/t/why-is-garbage-collection-called-here/109870/10 "2024-02-08T10:17:56Z")

</div>

So glad I could help you 😃

The final type instability caused by the type instability of the function `LoadParticlesFromCSV`. Because you load things from file the compiler cannot tell the types and so `DF_FLUID.Rhop` and `DF_BOUND.Rhop` are vectors of unknown type. You were saved from causing further type instability in the rest of the function because you copy these into an array of known type (`Density`).

To fix this, I would propose moving the loading of the files into the arguments of `RunSimulation` (and while we are doing this, we can also extract FloatType in a more idiomatic way). Here is a slightly modified start of `RunSimulation` with some comments:

```julia
function RunSimulation(;FluidCSV::String,
                        BoundCSV::String,
                        SimMetaData::SimulationMetaData{FloatType},
                        SimConstants::SimulationConstants,
                        (points,DF_FLUID,DF_BOUND) = LoadParticlesFromCSV(FluidCSV,BoundCSV)
) where FloatType
    # FloatType - this extracts the value inside of {} of the SimulationMetaData value
    # FloatType = typeof(SimMetaData).parameters[1]
    # we don't need this any more, if we extract the type directly from the args :)

    # Unpack the relevant simulation meta data
    @unpack HourGlass, SaveLocation, SimulationName, MaxIterations, OutputIteration, SilentOutput, ThreadsCPU = SimMetaData;

    # Unpack simulation constants
    @unpack ρ₀, dx, h, m₀, αD, α, g, c₀, γ, dt, δᵩ, CFL, η² = SimConstants

    # Load in the fluid and boundary particles. Return these points and both data frames
    # points,DF_FLUID,DF_BOUND = LoadParticlesFromCSV(FluidCSV,BoundCSV)
    # this was moved to the args for type stability

    # Generate simulation data results array
    FinalResults = SimulationDataResults{3,FloatType}(NumberOfParticles = length(points))
    @unpack Kernel, KernelGradient, Density, Position, Acceleration, Velocity = FinalResults
    # Initialize Arrays
    Position .= deepcopy(points)
    # note that this allocates 2 tempory arrays (i think)
    Density .= Array([DF_FLUID.Rhop;DF_BOUND.Rhop])
    # does this work as well?
    # Density .= [DF_FLUID.Rhop;DF_BOUND.Rhop]
    # or maybe like this (no temp allocation but this doesn't matter here)
    # Density[1:length(DF_FLUID.Rhop)] .= DF_FLUID.Rhop
    # Density[length(DF_FLUID.Rhop)+1:end] .= DF_BOUND.Rhop

    # rest of the function

```

---

<div class="post-metadata">

**Author:** ![Ahmed\_Salih](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ahmed_salih/32/206579_2.png) [@Ahmed\_Salih](https://discourse.julialang.org/u/Ahmed_Salih)\
**Post date:** [February 8, 2024, 10:54am UTC](https://discourse.julialang.org/t/why-is-garbage-collection-called-here/109870/11 "2024-02-08T10:54:17Z")

</div>

Hi again!

Thanks a ton, didn’t even know about `FloatType` inside of function signature with `where` clause, would work!

Got it, I ended up fixing density, by modifying the way I load particles in and preallocating results into a vector of type `FloatType`.

I was really confused though why something like:

`Density .= FloatType.([Arr1;Arr2])`

Would still give a type error? Is this due to it not knowing the types of Arr1 and Arr2 at compile time when it is from loading a CSV, so even forcefully converting to `FloatType` it cannot figure it out at compile time?

Kind regards

---

<div class="post-metadata">

**Author:** ![abraemer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/abraemer/32/51403_2.png) [@abraemer](https://discourse.julialang.org/u/abraemer)\
**Post date:** [February 8, 2024, 11:02am UTC](https://discourse.julialang.org/t/why-is-garbage-collection-called-here/109870/12 "2024-02-08T11:02:48Z")

</div>

The type instability is coming from `[Arr1; Arr2]`. This expression is lowered to `vcat(Arr1, Arr2)` and since the element types of `Arr1` and `Arr2` are unknown this causes a dynamic dispatch and the return type is unknown. After that you convert it forcefully to `FloatType` by writing the values to `Density` which has a known type. (I am actually not sure if `FloatType.(...)` is strong enough to guarantee the type, it is anyways unecessary as the assignment attempts a conversion anyways).

So the type instability is not contagious in this case. It’s only during the setup that there are some instabilities but the rest of the code is perfectly type stable. So I don’t think this instability must be fixed necessarily 🙂