# Ccall Fortran OpenBLAS multithreading issues with Julia 1.7

**URL:** <https://discourse.julialang.org/t/ccall-fortran-openblas-multithreading-issues-with-julia-1-7/72913>\
**Category:** Julia at Scale\
**Tags:** blas\
**Created:** [December 10, 2021, 9:53pm UTC](https://discourse.julialang.org/t/ccall-fortran-openblas-multithreading-issues-with-julia-1-7/72913 "2021-12-10T21:53:56Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Perrin\_Meyer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/perrin_meyer/32/15686_2.png) [@Perrin\_Meyer](https://discourse.julialang.org/u/Perrin_Meyer)\
**Post date:** [December 10, 2021, 9:53pm UTC](https://discourse.julialang.org/t/ccall-fortran-openblas-multithreading-issues-with-julia-1-7/72913/1 "2021-12-10T21:53:56Z")

</div>

I am using Julia to drive a Fortran based Boundary Element code. The Fortran code uses OpenMP and OpenBLAS for multithreading.

**When I call into the BEM code from Julia 1.7, I now only get single-threaded performance.**

On my previous computer with Julia 1.6, I was able to use CCALL to call into the Fortran code and get the same multithreading performance as when I compiled a stand-alone Fortan driver/executable. I was not setting any environment variables, it just “worked”.

**I know its an OpenBLAS interaction** , since If I add

`using MKL`

to the top of my Julia driver, then when I CCALL into the Fortran code I get the same multithreading performance as a stand-alone Fortran driver.

**what are the magic incantations (environment variables?..)** to tell my Fortran/OpenBLAS code to use all 48 threads (in this case, Julia is only the driver, so I don’t care if Julia is single threaded, although it would be great if **both** could be multithreaded…

When I compile OpenBLAS for the Fortran BEM library, I use

`make USE_OPENMP=1 DYNAMIC_ARCH=1 NUM_THREADS=48`

Is this the correct incantation for compiling OpenBLAS to “play nice” with Julia Multithreading?

In theory, should I try to link my Fortran code to the same OpenBLAS library Julia uses? or is that a bad idea?

Thanks

---

<div class="post-metadata">

**Author:** ![Perrin\_Meyer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/perrin_meyer/32/15686_2.png) [@Perrin\_Meyer](https://discourse.julialang.org/u/Perrin_Meyer)\
**Post date:** [December 11, 2021, 3:53am UTC](https://discourse.julialang.org/t/ccall-fortran-openblas-multithreading-issues-with-julia-1-7/72913/2 "2021-12-11T03:53:03Z")

</div>

Or maybe

`using MKL`

sets the Environment variables and threading parameters “correctly”, so its not necessarily an OpenBLAS interaction…

googling “openBLAS” and Julia brings up the fascinating discussion of libblastrampoline, which gives me the impression that OpenMP and OpenBLAS and BLAS libraries in general are really messy (especially trying support multiple hardware/OS combinations).

So any advice or pointers appreciated…

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [December 11, 2021, 6:35am UTC](https://discourse.julialang.org/t/ccall-fortran-openblas-multithreading-issues-with-julia-1-7/72913/3 "2021-12-11T06:35:21Z")

</div>

Try setting `OMP_NUM_THREADS=48`.

---

<div class="post-metadata">

**Author:** ![Perrin\_Meyer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/perrin_meyer/32/15686_2.png) [@Perrin\_Meyer](https://discourse.julialang.org/u/Perrin_Meyer)\
**Post date:** [December 11, 2021, 11:54pm UTC](https://discourse.julialang.org/t/ccall-fortran-openblas-multithreading-issues-with-julia-1-7/72913/4 "2021-12-11T23:54:20Z")

</div>

unfortunately, neither  
`ENV["OMP_NUM_THREADS"]=48`  
at the top of the Julia driver nor  
`export OMP_NUM_THREADS=48`  
in the shell  
restore multi-threading performance  
However, putting  
`using MKL`  
at the top of the Julia driver code does restore multithreading (top goes from 99% cpu utilization to 3980% utilization during the mostly-embarrassingly-parallel quadrature phase of the BEM code. )

Any other suggestions for debugging this greatly appreciated. `Using MKL `is certainly a fine stopgap solution, but I’d like to figure out how to make OpenBLAS/OpenMP happy if possible.

thanks

---

<div class="post-metadata">

**Author:** ![Ralph\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ralph_smith/32/10344_2.png) [@Ralph\_Smith](https://discourse.julialang.org/u/Ralph_Smith)\
**Post date:** [December 11, 2021, 11:55pm UTC](https://discourse.julialang.org/t/ccall-fortran-openblas-multithreading-issues-with-julia-1-7/72913/5 "2021-12-11T23:55:46Z")

</div>

If your Fortran library only uses the standard BLAS/LAPACK API (and not OpenBLAS internals) you should be able to build it so that it simply links to the `libblastrampoline.so` in your Julia system instead of OpenBLAS. Then threading should be handled consistently and (fairly) safely, and there would be other advantages. Let’s ask @staticfloat if there are known problems with this approach.

---

<div class="post-metadata">

**Author:** ![staticfloat](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/staticfloat/32/34_2.png) [@staticfloat](https://discourse.julialang.org/u/staticfloat)\
**Post date:** [December 12, 2021, 1:49pm UTC](https://discourse.julialang.org/t/ccall-fortran-openblas-multithreading-issues-with-julia-1-7/72913/6 "2021-12-12T13:49:47Z")

</div>

Linking directly against LBT is fine, but it should mostly help with being BLAS vendor independent, more than working well with threads.

Can you try BLAS.set\_num\_threads() and see if changing that does anything? I wonder if they version of OpenBLAS we ship in 1.7 has different multi threading thresholds and just doesn’t think your problem is worth splitting up into chunks. Can you try invoking a giant square GEMM or something like that just to ensure that it’s not due to OpenBLAS not wanting to multithread a really skinny matrix?

---

<div class="post-metadata">

**Author:** ![Perrin\_Meyer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/perrin_meyer/32/15686_2.png) [@Perrin\_Meyer](https://discourse.julialang.org/u/Perrin_Meyer)\
**Post date:** [December 13, 2021, 7:48pm UTC](https://discourse.julialang.org/t/ccall-fortran-openblas-multithreading-issues-with-julia-1-7/72913/7 "2021-12-13T19:48:01Z")

</div>

`BLAS.set_num_threads(48)`  
does not help.

For this Fortran driver, I’m not concerned about Julia BLAS multi-threading, its just that when I ccall() into the BEM fortran library, I get only single threading performance in the Fortran library, unless I put  
`using MKL`  
at the top of my Julia driver file, in which case I get the same OpenMP/OpenBLAS multithreading performance as when I compile a stand-alone Fortran executable.

Since I’m linking the Fortran library to a OpenBLAS library I’m compiling from source, the only thing I can think is that the Julia OpenBLAS/OpenMP implementation is fighting with the Fortran OpenBLAS/OpenMP threading…

---

<div class="post-metadata">

**Author:** ![staticfloat](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/staticfloat/32/34_2.png) [@staticfloat](https://discourse.julialang.org/u/staticfloat)\
**Post date:** [December 14, 2021, 4:06pm UTC](https://discourse.julialang.org/t/ccall-fortran-openblas-multithreading-issues-with-julia-1-7/72913/8 "2021-12-14T16:06:28Z")

</div>

Can you try invoking a giant square GEMM or something like that just to ensure that it’s not due to OpenBLAS not wanting to multithread a really skinny matrix?