# Resource usage in versions 1.7.2 and 1.8.0

**URL:** <https://discourse.julialang.org/t/resource-usage-in-versions-1-7-2-and-1-8-0/86111>\
**Category:** Performance\
**Created:** [August 22, 2022, 1:48am UTC](https://discourse.julialang.org/t/resource-usage-in-versions-1-7-2-and-1-8-0/86111 "2022-08-22T01:48:35Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![marllos](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marllos/32/205917_2.png) [@marllos](https://discourse.julialang.org/u/marllos)\
**Post date:** [August 22, 2022, 1:48am UTC](https://discourse.julialang.org/t/resource-usage-in-versions-1-7-2-and-1-8-0/86111/1 "2022-08-22T01:48:35Z")

</div>

Hello everyone. I updated Julia from version 1.7.2 to 1.8.0 and had a surprise: a code that used 2 GB of RAM, now uses 10 GB, and the number of subprocesses (htop) also increased from 8 to 64! Is this normal? I was a bit worried.

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [August 22, 2022, 2:04am UTC](https://discourse.julialang.org/t/resource-usage-in-versions-1-7-2-and-1-8-0/86111/2 "2022-08-22T02:04:01Z")

</div>

That’s really concerning. Any chance you can give an example that reproduces the behavior? is there a corresponding speed increase? Also, how many cores does your system have?

---

<div class="post-metadata">

**Author:** ![marllos](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marllos/32/205917_2.png) [@marllos](https://discourse.julialang.org/u/marllos)\
**Post date:** [August 22, 2022, 2:26am UTC](https://discourse.julialang.org/t/resource-usage-in-versions-1-7-2-and-1-8-0/86111/3 "2022-08-22T02:26:33Z")

</div>

Whenever I can, I make sample code, but in this case there is no way, because it is a big code. I still don’t know if it is more efficient, because the execution time varies a lot, between 12 and 24 hours. The system is a 64-core Ubuntu 20.04 server. On my notebook, with 4 cores and 8 GB of RAM this increase did not occur.

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [August 22, 2022, 2:50am UTC](https://discourse.julialang.org/t/resource-usage-in-versions-1-7-2-and-1-8-0/86111/4 "2022-08-22T02:50:03Z")

</div>

Are you using a lot of matrix multiplies? I believe in 1.8 we increased the maximum number of BLAS threads.

---

<div class="post-metadata">

**Author:** ![marllos](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marllos/32/205917_2.png) [@marllos](https://discourse.julialang.org/u/marllos)\
**Post date:** [August 22, 2022, 2:59am UTC](https://discourse.julialang.org/t/resource-usage-in-versions-1-7-2-and-1-8-0/86111/5 "2022-08-22T02:59:47Z")

</div>

Yes, I use the LinearAlgebra package a lot and make many products of ComplexF64 Hermitian matrices of dimensions 8x8x2x1000 and other smaller dimensions.

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [August 22, 2022, 3:09am UTC](https://discourse.julialang.org/t/resource-usage-in-versions-1-7-2-and-1-8-0/86111/6 "2022-08-22T03:09:02Z")

</div>

Try lowering the number of BLAS threads. (or if you want your program to be faster use MKL)

---

<div class="post-metadata">

**Author:** ![marllos](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marllos/32/205917_2.png) [@marllos](https://discourse.julialang.org/u/marllos)\
**Post date:** [August 22, 2022, 3:16am UTC](https://discourse.julialang.org/t/resource-usage-in-versions-1-7-2-and-1-8-0/86111/7 "2022-08-22T03:16:06Z")

</div>

Interesting: BLAS threads 1.7.2 = 8; 1.8.0 = 32. I’ll test it tomorrow and get back to you.

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [August 22, 2022, 5:40am UTC](https://discourse.julialang.org/t/resource-usage-in-versions-1-7-2-and-1-8-0/86111/8 "2022-08-22T05:40:58Z")

</div>

Instead of just lowering the number of BLAS threads I recommend that you benchmark your code properly to see if using more BLAS threads gives you better performance (it probably should) as @Oscar_Smith said before. After all, if you have a 64 core system you typically don’t want to only use a small fraction of the cores.

Also, guessing from the number of cores (64), does this happen to be an AMD system? If yes, you might want to try [BLISBLAS.jl](https://github.com/carstenbauer/BLISBLAS.jl) or [MKL.jl](https://github.com/JuliaLinearAlgebra/MKL.jl), see [GitHub - carstenbauer/julia-dgemm-noctua: DGEMM Benchmarks on Noctua 1/2 at PC2](https://github.com/carstenbauer/julia-dgemm-noctua#noctua-2).

---

<div class="post-metadata">

**Author:** ![marllos](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marllos/32/205917_2.png) [@marllos](https://discourse.julialang.org/u/marllos)\
**Post date:** [August 22, 2022, 1:58pm UTC](https://discourse.julialang.org/t/resource-usage-in-versions-1-7-2-and-1-8-0/86111/9 "2022-08-22T13:58:34Z")

</div>

I used the command BLAS.set\_num\_threads(8) right after using LinearAlgebra, and it had no effect.

---

<div class="post-metadata">

**Author:** ![marllos](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marllos/32/205917_2.png) [@marllos](https://discourse.julialang.org/u/marllos)\
**Post date:** [August 22, 2022, 1:59pm UTC](https://discourse.julialang.org/t/resource-usage-in-versions-1-7-2-and-1-8-0/86111/10 "2022-08-22T13:59:05Z")

</div>

Each run lasts an average of 18 hours, so it is hard to benchmark, but I will try anyway. I think it is great when I can use many cores, but I can’t always, because of other users.

---

<div class="post-metadata">

**Author:** ![marllos](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marllos/32/205917_2.png) [@marllos](https://discourse.julialang.org/u/marllos)\
**Post date:** [August 22, 2022, 2:01pm UTC](https://discourse.julialang.org/t/resource-usage-in-versions-1-7-2-and-1-8-0/86111/11 "2022-08-22T14:01:13Z")

</div>

Julia Version 1.8.0  
Commit 5544a0fab76 (2022-08-17 13:38 UTC)  
Platform Info:  
OS: Linux (x86\_64-linux-gnu)  
CPU: 64 × AMD EPYC 7452 32-Core Processor  
WORD\_SIZE: 64  
LIBM: libopenlibm  
LLVM: libLLVM-13.0.1 (ORCJIT, znver2)  
Threads: 1 on 64 virtual cores

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [August 22, 2022, 2:01pm UTC](https://discourse.julialang.org/t/resource-usage-in-versions-1-7-2-and-1-8-0/86111/12 "2022-08-22T14:01:21Z")

</div>

For initial testing, you should probably start by just testing the speed of complex hermitian matrix multiplication.

---

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [August 22, 2022, 2:20pm UTC](https://discourse.julialang.org/t/resource-usage-in-versions-1-7-2-and-1-8-0/86111/13 "2022-08-22T14:20:50Z")

</div>

Could you try in Julia 1.9-DEV nightly? E.g. with:

```julia
julia --heap-size-hint=2GB

```

I would at least be intrigued to know if this new option, from 1.9 NEWS, helps:

> New option `--heap-size-hint=<size>` gives a memory hint for triggering greedy garbage collection. The size might be specified in bytes, kilobytes(1000k), megabytes(300M), gigabytes(1.5G)

The input and output is the same (size), only some temporary intermediate computation larger, so this should help? Eventually with better GC wouldn’t be needed, and also we will all have 10 GB+ in the future, it will be peanuts…

---

<div class="post-metadata">

**Author:** ![digital\_carver](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/digital_carver/32/33818_2.png) [@digital\_carver](https://discourse.julialang.org/u/digital_carver)\
**Post date:** [August 22, 2022, 3:08pm UTC](https://discourse.julialang.org/t/resource-usage-in-versions-1-7-2-and-1-8-0/86111/14 "2022-08-22T15:08:33Z")

</div>

From the text you posted from NEWS.md, it seems like it should be

```julia
julia --heap-size-hint=2G

```

(i.e. without the `B`)

---

<div class="post-metadata">

**Author:** ![marllos](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marllos/32/205917_2.png) [@marllos](https://discourse.julialang.org/u/marllos)\
**Post date:** [August 22, 2022, 3:23pm UTC](https://discourse.julialang.org/t/resource-usage-in-versions-1-7-2-and-1-8-0/86111/15 "2022-08-22T15:23:44Z")

</div>

And where do I download Julia 1.9-DEV nightly, I download it from  
Nightly builds but it didn’t work, I got the error:  
/usr/local/bin/julia301b62ae73: cannot execute binary file: Exec format error

---

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [August 23, 2022, 8:56am UTC](https://discourse.julialang.org/t/resource-usage-in-versions-1-7-2-and-1-8-0/86111/16 "2022-08-23T08:56:24Z")

</div>

Maybe missing chmod, or see here:

> **[How to fix Cannot execute binary file: exec format error? | Candid.Technology](https://candid.technology/cannot-execute-binary-file-exec-format-error/)**
>
> In this article, we're taking a look at the "cannot execute binary file: exec format error" problem in Linux and giving you some solutions.

Did you for sure download the right binary?

> [@digital\_carver](#):
>
> (i.e. without the `B`)

My somewhat older 1.9-DEV ~~worked~~ didn’t complain with:

```julia
julia --heap-size-hint=2GB

```

But it’s a good catch, it might not work if I’m reading the code correctly:

> <https://github.com/JuliaLang/julia/blob/bd26069cee94bda2b7dec498a6bf92fb2aff76d7/src/jloptions.c#L770>

Does it actually parse as B (only last letter), just hitting the default/break case and then “2GB” = “2” (bytes)?

I found it likely to be intentional and equivalent (but I didn’t check if it was actually did the same or anything, GXXX also runs and XXX), but 1.9 isn’t released, and if G is in the spec, and GB isn’t, do not rely on it to work ever… It could be changed to work before release, or not (or even changed back to spec, by accident, though unlikely).

---

<div class="post-metadata">

**Author:** ![marllos](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marllos/32/205917_2.png) [@marllos](https://discourse.julialang.org/u/marllos)\
**Post date:** [August 23, 2022, 5:40pm UTC](https://discourse.julialang.org/t/resource-usage-in-versions-1-7-2-and-1-8-0/86111/17 "2022-08-23T17:40:36Z")

</div>

I had installed the wrong architecture. Now it’s right:  
Julia Version 1.9.0-DEV.1172  
Commit 18fa3835a78 (2022-08-23 13:44 UTC)

Using the --heap-size-hint=2GB or --heap-size-hint=2G option, it still shows in htop-VIRT = 9.6G instead of 2.0G as before. I also tested with and without BLAS.set\_num\_threads(8).

Julia also runs  
julia --project=.juliadev --heap-size-hint=2any\_string

---

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [August 26, 2022, 3:46pm UTC](https://discourse.julialang.org/t/resource-usage-in-versions-1-7-2-and-1-8-0/86111/18 "2022-08-26T15:46:46Z")

</div>

You may need, or want to file an issue on this.

---

<div class="post-metadata">

**Author:** ![marllos](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marllos/32/205917_2.png) [@marllos](https://discourse.julialang.org/u/marllos)\
**Post date:** [August 26, 2022, 7:43pm UTC](https://discourse.julialang.org/t/resource-usage-in-versions-1-7-2-and-1-8-0/86111/19 "2022-08-26T19:43:45Z")

</div>

Yes, I want to, but I don’t know how to do it.

---

<div class="post-metadata">

**Author:** ![jbytecode](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jbytecode/32/17719_2.png) [@jbytecode](https://discourse.julialang.org/u/jbytecode)\
**Post date:** [August 30, 2022, 7:58pm UTC](https://discourse.julialang.org/t/resource-usage-in-versions-1-7-2-and-1-8-0/86111/20 "2022-08-30T19:58:54Z")

</div>

Providing a minimal example of bloating memory usage would be great.