# Scaling of @threads for "embarrassingly parallel" problem
**URL:** https://discourse.julialang.org/t/scaling-of-threads-for-embarrassingly-parallel-problem/92949
**Category:** Performance
**Tags:** threads
**Created:** [January 14, 2023, 4:08am UTC](https://discourse.julialang.org/t/scaling-of-threads-for-embarrassingly-parallel-problem/92949 "2023-01-14T04:08:00Z")
**Posts on this page:** 10
**Page:** 2
### Author:  [@miguelraz](https://discourse.julialang.org/u/miguelraz)
#### Post date: [January 16, 2023, 5:45am UTC](https://discourse.julialang.org/t/scaling-of-threads-for-embarrassingly-parallel-problem/92949/21 "2023-01-16T05:45:43Z")
For future reference of other readers here:
Everyone who starts in a parallel computing course is told the golden rule to first squeeze the most performance out of your serial code and then to parallelize that.
Allocations have to do syscalls which inevitably hand over control to the operating system at unpredictable moments and are thus a problem in hot loops.
I agree that tooling can be improved to specify faster where such allocations happen, but perhaps this `@deallocate` path is not the best way forward.
---
### Author:  [@martin.d.maas](https://discourse.julialang.org/u/martin.d.maas)
#### Post date: [January 17, 2023, 4:30pm UTC](https://discourse.julialang.org/t/scaling-of-threads-for-embarrassingly-parallel-problem/92949/22 "2023-01-17T16:30:45Z")
> [@goerz](#):
>
> The whole issue does leave a bad aftertaste, though: I really feel that Julia’s garbage collector should handle this situation much better. At this point, garbage collection is the number one worry I have about Julia’s efficacy. Maybe in future versions, (in addition to just fixing the GC in parallel threads), Julia could gain a built-in `@deallocate` for manual memory management, which would allow turning off garbage collection for large section of a program — although I understand that this inevitably opens up the door to potential segfaults.
I usually share the same kind of concerns. However, its not clear to me that the main problem here (or, in general) is the GC per-se, as the % of time spent on it was low.
You’ve also reduced the allocations from 4Gb to 500 MB which is a big deal. And your single-threaded performance seems to have improved significantly (from 250s to 54s, if you where indeed running the same test case). So to me, this seems to have been a problem with memory usage and memory bandwith.
Julia, by default, allocates arrays on the heap and can create copies of arrays when you don’t expect them. A general method to reduce (costly) allocations would be to use stack-allocated arrays, which is however, not so easy to do in Julia.
So I also wish working with stack-allocated arrays (which, in particular, might require manual memory management) were more straightforward in Julia. I’m currently experimenting with StrideArrays.jl and Bumper.jl, which might well be the solution to this problem so we don’t need to wait for Julia 2.0.
---
### Author:  [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)
#### Post date: [January 17, 2023, 4:42pm UTC](https://discourse.julialang.org/t/scaling-of-threads-for-embarrassingly-parallel-problem/92949/23 "2023-01-17T16:42:22Z")
Allocations almost never have to do syscalls (they do but only for fairly large allocations, and even then it should be somewhat rare).
---
### Author:  [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)
#### Post date: [January 17, 2023, 4:48pm UTC](https://discourse.julialang.org/t/scaling-of-threads-for-embarrassingly-parallel-problem/92949/24 "2023-01-17T16:48:50Z")
Arrays on the stack vs heap doesn’t matter. They’re both just memory. What does matter is how they are allocated and cleaned up. One specific thing Julia could do that would help a bunch is have better analysis for array lifetimes which would let the compiler replace gcing and allocating a new array with just reusing the memory (but doing this isn’t trivial.
---
### Author:  [@Salmon](https://discourse.julialang.org/u/Salmon)
#### Post date: [January 17, 2023, 6:23pm UTC](https://discourse.julialang.org/t/scaling-of-threads-for-embarrassingly-parallel-problem/92949/25 "2023-01-17T18:23:16Z")
I wonder, if each thread allocates memory, can there be slowdowns since it needs to be made sure that they don’t allocate the same memory twice? Never heard whether that’s a thing but this made me curious
---
### Author:  [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)
#### Post date: [January 17, 2023, 6:41pm UTC](https://discourse.julialang.org/t/scaling-of-threads-for-embarrassingly-parallel-problem/92949/26 "2023-01-17T18:41:07Z")
Multi-threaded allocators aren’t that complicated. The TLDR is that each thread has it’s own pool of pages into which it can allocate small objects. For big objects it will call into the system allocator and there will be some overhead, but it’s relatively minor compared to the cost of actually using that memory for anything.
---
### Author:  [@martin.d.maas](https://discourse.julialang.org/u/martin.d.maas)
#### Post date: [January 17, 2023, 6:56pm UTC](https://discourse.julialang.org/t/scaling-of-threads-for-embarrassingly-parallel-problem/92949/27 "2023-01-17T18:56:45Z")
> [@Oscar\_Smith](#):
>
> Arrays on the stack vs heap doesn’t matter. They’re both just memory
But there are different kinds of memory: cache memory and RAM. In my somewhat limited experience, I tend to think (I’m a practitioner, not a compiler developer) that whenever I manage to allocate a small amount of memory to the stack, the compiler figures out it should use the cache.
> [@Oscar\_Smith](#):
>
> One specific thing Julia could do that would help a bunch is have better analysis for array lifetimes which would let the compiler replace gcing and allocating a new array with just reusing the memory (but doing this isn’t trivial.
Sure, that would be great. But maybe we could help it in the meantime with some allocate/deallocate for performance-critical sections of code. I still haven’t found a nice way to do so.
---
### Author:  [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)
#### Post date: [January 17, 2023, 7:14pm UTC](https://discourse.julialang.org/t/scaling-of-threads-for-embarrassingly-parallel-problem/92949/28 "2023-01-17T19:14:48Z")
Cache isn’t managed explicitly by the compiler. All cache decisions are made by the CPU. Memory on the stack often ends up in cache because it is frequently referenced, but heap memory will also do that.
In general rather than manually allocating/deallocating you can get the same (or slightly better) advantage from just reusing the memory and not reallocating in the first place.
---
### Author:  [@miguelraz](https://discourse.julialang.org/u/miguelraz)
#### Post date: [January 19, 2023, 9:55pm UTC](https://discourse.julialang.org/t/scaling-of-threads-for-embarrassingly-parallel-problem/92949/29 "2023-01-19T21:55:15Z")
Ah, my bad, thanks for the correction.
---
### Author:  [@mkitti](https://discourse.julialang.org/u/mkitti)
#### Post date: [January 20, 2023, 9:01am UTC](https://discourse.julialang.org/t/scaling-of-threads-for-embarrassingly-parallel-problem/92949/30 "2023-01-20T09:01:22Z")
> [@goerz](#):
>
> The whole issue does leave a bad aftertaste, though: I really feel that Julia’s garbage collector should handle this situation much better. At this point, garbage collection is the number one worry I have about Julia’s efficacy. Maybe in future versions, (in addition to just fixing the GC in parallel threads), Julia could gain a built-in `@deallocate` for manual memory management, which would allow turning off garbage collection for large section of a program — although I understand that this inevitably opens up the door to potential segfaults.
It is possible to run GC manually and also disable and enable it today. You can even use `Libc.malloc` and `Libc.free` to manually allocate and free memory. If you need an array, you use `unsafe_wrap`.
```julia
help?> GC.enable
GC.enable(on::Bool)
Control whether garbage collection is enabled using a boolean argument (true for enabled, false for disabled). Return previous GC state.
│ Warning
│
│ Disabling garbage collection should be used only with caution, as it can cause memory use to grow without bound.
help?> GC.gc
GC.gc([full=true])
Perform garbage collection. The argument full determines the kind of collection: A full collection (default) sweeps all objects, which makes the next GC scan much slower, while an incremental
collection may only sweep so-called young objects.
│ Warning
│
│ Excessive use will likely lead to poor performance.
```
None of that will really address the core issue of excessive allocations though. Fortunately, Julia offers pretty nice constructs and tools to help you avoid allocations.
[Previous page](https://discourse.julialang.org/t/scaling-of-threads-for-embarrassingly-parallel-problem/92949.md?page=1)