# Memory usage and performance differences in 1.9-rc1 vs. 1.8.5

**URL:** <https://discourse.julialang.org/t/memory-usage-and-performance-differences-in-1-9-rc1-vs-1-8-5/96209>\
**Category:** Internals & Design\
**Created:** [March 16, 2023, 9:09pm UTC](https://discourse.julialang.org/t/memory-usage-and-performance-differences-in-1-9-rc1-vs-1-8-5/96209 "2023-03-16T21:09:33Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![BioTurboNick](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bioturbonick/32/6380_2.png) [@BioTurboNick](https://discourse.julialang.org/u/BioTurboNick)\
**Post date:** [March 16, 2023, 9:09pm UTC](https://discourse.julialang.org/t/memory-usage-and-performance-differences-in-1-9-rc1-vs-1-8-5/96209/1 "2023-03-16T21:09:33Z")

</div>

I have some memory- and CPU-intensive multithreaded image analysis code.

In 1.8.5, on an AWS Linux system with 64 GB of RAM and 8 vCPUs, the code will run in cycles, spiking up to (for one example) 23 GB of allocated RAM and 800% CPU usage, executing in 37 min. Running the GC at the end drops memory usage to 7 GB, and then with malloc\_trim, 3.0 GB.

On 1.9-rc1, I observe no more than 600% CPU usage, no more than 7 GB allocated (in a second phase that isn’t parallel, goes up to 14 GB), executing in 33 min. Running the GC at the end drops memory usage to 5.0 GB, and then with malloc\_trim, 2.9 GB.

May be related to @Paul_Soderlind 's observation here: [1.9-rc1 and threads](https://discourse.julialang.org/t/1-9-rc1-and-threads/96119)

---

<div class="post-metadata">

**Author:** ![mbauman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mbauman/32/31082_2.png) [@mbauman](https://discourse.julialang.org/u/mbauman)\
**Post date:** [March 16, 2023, 9:17pm UTC](https://discourse.julialang.org/t/memory-usage-and-performance-differences-in-1-9-rc1-vs-1-8-5/96209/2 "2023-03-16T21:17:35Z")

</div>

So by absolutely every metric, Julia 1.9 is better? That’s great.

---

<div class="post-metadata">

**Author:** ![BioTurboNick](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bioturbonick/32/6380_2.png) [@BioTurboNick](https://discourse.julialang.org/u/BioTurboNick)\
**Post date:** [March 16, 2023, 10:25pm UTC](https://discourse.julialang.org/t/memory-usage-and-performance-differences-in-1-9-rc1-vs-1-8-5/96209/3 "2023-03-16T22:25:49Z")

</div>

Why isn’t multithreaded code saturating the CPUs anymore though?

---

<div class="post-metadata">

**Author:** ![simsurace](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/simsurace/32/30216_2.png) [@simsurace](https://discourse.julialang.org/u/simsurace)\
**Post date:** [March 17, 2023, 9:23am UTC](https://discourse.julialang.org/t/memory-usage-and-performance-differences-in-1-9-rc1-vs-1-8-5/96209/4 "2023-03-17T09:23:19Z")

</div>

I wonder about this as well. While the result is faster overall, it seems that Julia 1.9 still leaves 25% of performance on the table in this example.

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [March 17, 2023, 9:32am UTC](https://discourse.julialang.org/t/memory-usage-and-performance-differences-in-1-9-rc1-vs-1-8-5/96209/5 "2023-03-17T09:32:21Z")

</div>

It’s hard to diagnose that remotely without source code. It’s quite possible that a lot of time in 1.8.5 was spent in the kernel for allocations, which is reduced by fewer allocations in 1.9, leading to an overall reduction in CPU usage.

---

<div class="post-metadata">

**Author:** ![cjdoris](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cjdoris/32/213133_2.png) [@cjdoris](https://discourse.julialang.org/u/cjdoris)\
**Post date:** [March 17, 2023, 4:58pm UTC](https://discourse.julialang.org/t/memory-usage-and-performance-differences-in-1-9-rc1-vs-1-8-5/96209/6 "2023-03-17T16:58:31Z")

</div>

If you were fully utilising your CPUs before and now you’re not, then your code is no longer CPU bounded - which is good because it means Julia has generated machine code which uses the CPU more efficiently.

Perhaps it’s now memory bandwidth bounded, given how much memory you’re using. If you want to go even faster you may need to improve your memory access patterns.

---

<div class="post-metadata">

**Author:** ![BioTurboNick](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bioturbonick/32/6380_2.png) [@BioTurboNick](https://discourse.julialang.org/u/BioTurboNick)\
**Post date:** [March 17, 2023, 9:39pm UTC](https://discourse.julialang.org/t/memory-usage-and-performance-differences-in-1-9-rc1-vs-1-8-5/96209/7 "2023-03-17T21:39:37Z")

</div>

`strace` doesn’t show many significant differences in system calls, except 40% fewer `futex` calls in 1.9-rc1, 30% of which error (proportion the same as 1.8.5)

What change in Julia could explain a switch from CPU-bound to memory-bound? [julia/NEWS.md at v1.9.0-rc1 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/blob/v1.9.0-rc1/NEWS.md)

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [March 20, 2023, 2:24pm UTC](https://discourse.julialang.org/t/memory-usage-and-performance-differences-in-1-9-rc1-vs-1-8-5/96209/8 "2023-03-20T14:24:57Z")

</div>

You wouldn’t see a difference in syscalls, since that’s the part of the workload that involves interacting with the rest of the world, which presumably hasn’t changed. Julia generating faster machine code could account for work getting done more efficiently and there not being enough of it to saturate more than six cores anymore. How well parallelized is your work?

There’s always the option to spin up two tamales that busy wait if you really want to see those last two cores pegged 😁

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [March 20, 2023, 2:28pm UTC](https://discourse.julialang.org/t/memory-usage-and-performance-differences-in-1-9-rc1-vs-1-8-5/96209/9 "2023-03-20T14:28:54Z")

</div>

Certainly 25% of performance is left on the table, but it may or may not be Julia that’s leaving it. The first suspect is the code: if it doesn’t expose sufficient parallelism then nothing the language does can fix that. If it does expose sufficient parallelism, the you can start looking at the language.

---

<div class="post-metadata">

**Author:** ![BioTurboNick](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bioturbonick/32/6380_2.png) [@BioTurboNick](https://discourse.julialang.org/u/BioTurboNick)\
**Post date:** [March 20, 2023, 3:02pm UTC](https://discourse.julialang.org/t/memory-usage-and-performance-differences-in-1-9-rc1-vs-1-8-5/96209/10 "2023-03-20T15:02:40Z")

</div>

Thanks all, I’ll take a look deeper into the code.

---

<div class="post-metadata">

**Author:** ![Paul\_Soderlind](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/paul_soderlind/32/1753_2.png) [@Paul\_Soderlind](https://discourse.julialang.org/u/Paul_Soderlind)\
**Post date:** [March 20, 2023, 3:35pm UTC](https://discourse.julialang.org/t/memory-usage-and-performance-differences-in-1-9-rc1-vs-1-8-5/96209/11 "2023-03-20T15:35:55Z")

</div>

Any chance you could come up with something that is portable? (I had doubts along similar lines, but failed to create something that others could easily run.)

---

<div class="post-metadata">

**Author:** ![BioTurboNick](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bioturbonick/32/6380_2.png) [@BioTurboNick](https://discourse.julialang.org/u/BioTurboNick)\
**Post date:** [March 25, 2023, 12:19am UTC](https://discourse.julialang.org/t/memory-usage-and-performance-differences-in-1-9-rc1-vs-1-8-5/96209/12 "2023-03-25T00:19:47Z")

</div>

I’ve been profiling to see what’s going on.

Looks like array copying (inside `collect`) is faster, sorting (inside `median`) is faster, `imfilter` is faster.

However I now see a substantial amount of time spent in runtime dispatch into the `StaticArray` constructor.

EDIT: And the allocation profiler shows that `StaticArray` is allocating `Core.SimpleVector`s… wat. Issue here: [1.9-rc1 regression - `StaticArray` allocates `Core.SimpleVector` objects, runtime dispatch · Issue #49145 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/issues/49145)

---

<div class="post-metadata">

**Author:** ![BioTurboNick](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bioturbonick/32/6380_2.png) [@BioTurboNick](https://discourse.julialang.org/u/BioTurboNick)\
**Post date:** [March 27, 2023, 4:12am UTC](https://discourse.julialang.org/t/memory-usage-and-performance-differences-in-1-9-rc1-vs-1-8-5/96209/13 "2023-03-27T04:12:42Z")

</div>

Turns out that in 1.9-rc1, there’s some sort of optimization failure in the case that an array length is passed into a type parameter.