# Is it possible to avoid these excess allocations when using @batch or @threads?

**URL:** <https://discourse.julialang.org/t/is-it-possible-to-avoid-these-excess-allocations-when-using-batch-or-threads/122208>\
**Category:** Performance\
**Tags:** memory-allocation, threads, closure, polyester\
**Created:** [November 3, 2024, 9:47pm UTC](https://discourse.julialang.org/t/is-it-possible-to-avoid-these-excess-allocations-when-using-batch-or-threads/122208 "2024-11-03T21:47:45Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Tetrakai](https://avatars.discourse-cdn.com/v4/letter/t/4da419/32.png) [@Tetrakai](https://discourse.julialang.org/u/Tetrakai)\
**Post date:** [November 3, 2024, 9:47pm UTC](https://discourse.julialang.org/t/is-it-possible-to-avoid-these-excess-allocations-when-using-batch-or-threads/122208/1 "2024-11-03T21:47:46Z")

</div>

It took some effort to come up with a short MWE for this issue. The goal is to avoid a few small allocations when mutating (an argument of an inner function) in parallel:

> **MWE**
>
> ```julia
> using BenchmarkTools, Parameters, Polyester, .Threads
> 
> # Composite-type struct
> @with_kw mutable struct Str
> A :: String = "A"
> res :: Vector{Int64} = fill(0, 10)
> end
> 
> ### Inner Function ###
> function addone(x, idx) 
> x[idx] + 1
> end
> 
> ### Outer Functions ###
> # Sequential
> function foo(n, str)
> for i in 1:n
> if i >= n
> str.res[i] = addone(str.res, i)
> end
> end
> end
> 
> # Polyester.@batch
> function foobatch(n, str)
> @batch for i in 1:n
> if i >= n
> str.res[i] = addone(str.res, i)
> end
> end
> end
> 
> # Threads.@threads
> function foothreads(n, str)
> @threads for i in 1:n
> if i >= n
> str.res[i] = addone(str.res, i)
> end
> end
> end
> 
> ### Benchmarks ###
> str = Str()
> @btime foo(10, x) setup = (x = deepcopy($str)) evals = 1
> @btime foobatch(10, x) setup = (x = deepcopy($str)) evals = 1
> @btime foothreads(10, x) setup = (x = deepcopy($str)) evals = 1
> 
> ### Check Results ###
> foo(10, str); str
> foobatch(10, str); str
> foothreads(10, str); str
> 
> ```

Benchmarks:

```julia
julia> @btime foo(10, x) setup = (x = deepcopy($str)) evals = 1
  20.000 ns (0 allocations: 0 bytes)

julia> @btime foobatch(10, x) setup = (x = deepcopy($str)) evals = 1
  1.410 μs (1 allocation: 32 bytes)

julia> @btime foothreads(10, x) setup = (x = deepcopy($str)) evals = 1
  34.601 μs (162 allocations: 16.81 KiB)

```

All functions mutate the (mutable) struct correctly:

> **Results**
>
> ```julia
> julia> foo(10, str); str
> Str
> A: String "A"
> res: Array{Int64}((10,)) [0, 0, 0, 0, 0, 0, 0, 0, 0, 1]
> 
> julia> foobatch(10, str); str
> Str
> A: String "A"
> res: Array{Int64}((10,)) [0, 0, 0, 0, 0, 0, 0, 0, 0, 2]
> 
> julia> foothreads(10, str); str
> Str
> A: String "A"
> res: Array{Int64}((10,)) [0, 0, 0, 0, 0, 0, 0, 0, 0, 3]
> 
> ```

The issue appears to be related to this topic, which never got fully resolved:

> [@\--track-allocation and @threads](https://discourse.julialang.org/t/track-allocation-and-threads/84729):
>
> Essentially, I call a function within nested loops. In serial, no heap allocations occur within the function. When run on multiple threads via @threads, allocations are reported within the function. Because the threaded case runs much slower, I suspect this is [not only a reporting issue](https://discourse.julialang.org/t/can-the-output-of-track-allocation-be-trusted-with-multi-threading-enabled/78744/2). I know there are some related issues to this elsewhere, but I haven’t really seen anything that would immediately solve this problem. In particular, it would be good to know whether this is something subtle…

Is it possible to avoid these? Using `@threads` would be ideal for my purposes, since `@batch` is slower when nested inside another parallel loop. But even the latter would be very informative/useful.

---

<div class="post-metadata">

**Author:** ![Gianluca\_Fuwa](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gianluca_fuwa/32/46071_2.png) [@Gianluca\_Fuwa](https://discourse.julialang.org/u/Gianluca_Fuwa)\
**Post date:** [November 3, 2024, 10:25pm UTC](https://discourse.julialang.org/t/is-it-possible-to-avoid-these-excess-allocations-when-using-batch-or-threads/122208/2 "2024-11-03T22:25:50Z")

</div>

I am not sure if there is a way to reduce the allocations with `@threads` unfortunately, but when using `@batch` allocations occur when non-`isbits` structs are inside the loop body, because it can not convert those to `PtrArrays` (at least as far as I understood it, but @Elrod probably knows more about that).

Therefore, in this case you can eliminate the single allocation by creating an alias to the `Vector` in `str` before the loop and using that one inside it:

```julia
function foobatch(n, str)
    res = str.res

    @batch for i in 1:n
        if i >= n
            res[i] = addone(res, i)
        end
    end
end

```

---

<div class="post-metadata">

**Author:** ![Tetrakai](https://avatars.discourse-cdn.com/v4/letter/t/4da419/32.png) [@Tetrakai](https://discourse.julialang.org/u/Tetrakai)\
**Post date:** [November 3, 2024, 11:12pm UTC](https://discourse.julialang.org/t/is-it-possible-to-avoid-these-excess-allocations-when-using-batch-or-threads/122208/3 "2024-11-03T23:12:27Z")

</div>

Thanks, that would explain it. Unfortunately the real code uses (roughly) an `MVector` of `structs` that are in turn of more `structs`. Those are finally isbits types (eg, `SVectors`), mostly.

The parallel loop is selecting elements of the `MVector` to pass on, so I don’t see how this solution could work without changing that and hurting the readability.

I’ll mark it as solved and maybe try rearranging things armed with this new knowledge.

---

<div class="post-metadata">

**Author:** ![Tetrakai](https://avatars.discourse-cdn.com/v4/letter/t/4da419/32.png) [@Tetrakai](https://discourse.julialang.org/u/Tetrakai)\
**Post date:** [November 8, 2024, 10:24am UTC](https://discourse.julialang.org/t/is-it-possible-to-avoid-these-excess-allocations-when-using-batch-or-threads/122208/4 "2024-11-08T10:24:42Z")

</div>

> [@Gianluca\_Fuwa](#):
>
> allocations occur when non-`isbits` structs

Do I understand correctly that a mutable struct is always non-isbits (even if all fields are isbits)? Thus I cannot avoid that small allocation when passing one to `@batch`.

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [November 8, 2024, 10:37am UTC](https://discourse.julialang.org/t/is-it-possible-to-avoid-these-excess-allocations-when-using-batch-or-threads/122208/5 "2024-11-08T10:37:37Z")

</div>

```julia
help?> isbitstype(A)
  isbitstype(T)

  Return true if type T is a "plain data" type, meaning it is immutable and contains no references to other values, only primitive types and other isbitstype types. [...]

```

So, yes.