# The improvement of memory management in multithreading Julia 1.10beta is amazing!

**URL:** <https://discourse.julialang.org/t/the-improvement-of-memory-management-in-multithreading-julia-1-10beta-is-amazing/102468>\
**Category:** General Usage\
**Tags:** multithreading, memory\
**Created:** [August 4, 2023, 2:06am UTC](https://discourse.julialang.org/t/the-improvement-of-memory-management-in-multithreading-julia-1-10beta-is-amazing/102468 "2023-08-04T02:06:25Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![liuyxpp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/liuyxpp/32/9870_2.png) [@liuyxpp](https://discourse.julialang.org/u/liuyxpp)\
**Post date:** [August 4, 2023, 2:06am UTC](https://discourse.julialang.org/t/the-improvement-of-memory-management-in-multithreading-julia-1-10beta-is-amazing/102468/1 "2023-08-04T02:06:25Z")

</div>

I am so happy and can not help to share here. I have a multithreading script that reads tons of CSV files, processes them, and saves each back to disk. before 1.10, I see a continuous growing of memory usage like insanely to eat up all 2T memory and eventually crashes with out-of-memory issue. Now with 1.10, the memory usage stabilizes at only 50G. What an enormous improvement!

---

<div class="post-metadata">

**Author:** ![algunion](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/algunion/32/51630_2.png) [@algunion](https://discourse.julialang.org/u/algunion)\
**Post date:** [August 4, 2023, 2:12am UTC](https://discourse.julialang.org/t/the-improvement-of-memory-management-in-multithreading-julia-1-10beta-is-amazing/102468/2 "2023-08-04T02:12:25Z")

</div>

Great. Let’s hope that `libuv` [issue](https://github.com/JuliaLang/julia/issues/50643) also gets fixed soon and the multithreading pain goes away forever.

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [August 4, 2023, 2:14am UTC](https://discourse.julialang.org/t/the-improvement-of-memory-management-in-multithreading-julia-1-10beta-is-amazing/102468/3 "2023-08-04T02:14:49Z")

</div>

I think some of the memory issues may be backported to 1.9. Our GC algorithm was previously somewhat “artisinal” and could make some pretty dumb decisions.

---

<div class="post-metadata">

**Author:** ![ericsfraga](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ericsfraga/32/192_2.png) [@ericsfraga](https://discourse.julialang.org/u/ericsfraga)\
**Post date:** [August 4, 2023, 8:39am UTC](https://discourse.julialang.org/t/the-improvement-of-memory-management-in-multithreading-julia-1-10beta-is-amazing/102468/4 "2023-08-04T08:39:24Z")

</div>

Thank you for posting this. It gives me hope that it might fix a problem we have been having (multi-threading with OpenModelica via OMJulia): memory consumption creeps up continuously until it exhausts the system memory of 128GB (not TB like you but still).

---

<div class="post-metadata">

**Author:** ![e3c6](https://avatars.discourse-cdn.com/v4/letter/e/e79b87/32.png) [@e3c6](https://discourse.julialang.org/u/e3c6)\
**Post date:** [August 4, 2023, 12:09pm UTC](https://discourse.julialang.org/t/the-improvement-of-memory-management-in-multithreading-julia-1-10beta-is-amazing/102468/5 "2023-08-04T12:09:52Z")

</div>

Looking forward to this. I too frequently have memory problems with multi-threaded code, possibly related to GC not doing its job.

> [@Oscar\_Smith](#):
>
> I think some of the memory issues may be backported to 1.9.

If 1.10 might take some time to be stable, then this would be very appreciated!

---

<div class="post-metadata">

**Author:** ![polyactis](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/polyactis/32/51034_2.png) [@polyactis](https://discourse.julialang.org/u/polyactis)\
**Post date:** [March 29, 2024, 2:16am UTC](https://discourse.julialang.org/t/the-improvement-of-memory-management-in-multithreading-julia-1-10beta-is-amazing/102468/6 "2024-03-29T02:16:06Z")

</div>

I am wondering what has caused this dramatic improvement?

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [March 29, 2024, 2:30am UTC](https://discourse.julialang.org/t/the-improvement-of-memory-management-in-multithreading-julia-1-10beta-is-amazing/102468/7 "2024-03-29T02:30:54Z")

</div>

The short answer is that GC tuning is a complicated tradeoff between time and space. Running GC more often keeps memory usage smaller, but takes more time. To make things more complicated, the runtime only knows when objects are allocated, not when they stop being used. As such the algorithms that trigger when to run GC are a pile of heuristics that if you get wrong, you end up using way too much ram or spending 90% of your time in GC.