Number of threads, puzzle

General Usage
, ,
1

draft.jl

c1 = Channel(100)
c2 = Channel(100)
c3 = Channel(100)

function task_1()
    while true
        wait(c1)
        take!(c1)
    end
end

function task_2()
    while true
        wait(c2)
        take!(c2)
    end
end

Threads.@spawn task_1()

Threads.@spawn task_2()

while true
    wait(c3)
    take!(c3)
end

On my 3 kernal 6 threads cpu, it will:
julia -q -t 1 – draft.jl 6 threads
julia -q -t 2 – draft.jl 8 threads
julia -q -t 3 – draft.jl 9 threads
julia -q -t 4 – draft.jl 10 threads

Why? Thanks!

2

I’m sorry but I don’t understand. What do you mean by this?

3

Thanks for your reply!
“julia -q -t 1 – draft.jl 6 threads” means there are “6 threads” in this new process, when it is started with “julia -q -t 1 – draft.jl”

4

Well it’s not just the “Julia threads” that get started. There’s also (at least) a signal thread (see signals-unix.c) as well as multiple OpenBLAS threads (libopenblas64_.so).

I’d suggest to do the following. First, use the built-in Threads.nthreads() to see that the number of Julia threads matches your expectation. Second, set OPENBLAS_NUM_THREADS=1 and rerun your original experiment to exclude the effect of OpenBLAS threads. Third, you may want something like done here and use gdb with a breakpoint at pthread_create to see which threads get started on a more fundamental level. Hope that helps.

UPDATE:

I just ran a simple experiment myself (with OPENBLAS_NUM_THREADS=1). I find

julia -t 1 # 2 threads
julia -t 2 # 3 threads
julia -t 3 # 4 threads
...

where I checked the number of threads with htop.

1 Like
5

Thank you very much!
I checked with htop too.
My julia version is “Version 1.6.4 (2021-11-19)”

I got the same result as yours, with OPENBLAS_NUM_THREADS=1.
Thanks again. From you I learn to know that there is another thread with all Channels, for the first time.

And where to learn more about details like above of this new language? I read the manual, but miss it.

6

These are internals and a user typically doesn’t have to care too much about an extra signal thread and such. (The number of Julia threads, i.e. what Threads.nthreads() gives, would have matched your expectation right away.)

However, where to find good information on this if one is interested / curious is a good question. Ideally in the developer section of the Julia docs, though I couldn’t find much on the unix signal thread in a quick search. Otherwise, as of right now, you either have to study the source code, debug with tools like gdb or, most importantly, ask questions here or on the Julia Slack I guess.

1 Like
7

Thanks a lot!

1 Like
8

I am recalling that a few months ago in one of discourse topics I allowed myself to suggest that a blog post on the topic of practical intricacies of multiprocessing / multithreading and pinning Julia threads with a focus on BLAS libraries could be potentially useful. I will allow myself to renew this suggestion with a hope that you, maybe even in a collaboration of some other Julia discourse colleagues, could potentially find some time to gather all your notes and provide (especially less advanced Julia users) with clear and practical information on those topics - I believe this could be useful.

1 Like
9

Not really what you have in mind I guess (it’s not very comprehensive) but FWIW I created Julia Threads + BLAS Threads · ThreadPinning.jl a few days ago. Maybe I or someone else finds the time to expand this further.

Also, at least the critical aspects of Julia BLAS interaction in case of multithreading probably needs to go into the Julia documentation itself. See @viralbshah 's comment on this here for example.

(I also tried to add a basic autochecking of MT settings feature to ThreadPinning.jl, see Autochecking BLAS Thread Settings · ThreadPinning.jl)

10

Thank you for pointing out to those new parts about BLAS Thread Settings at your GitHub repo/s. The first time I came by this problem was when trying to make AlphaZero.jl work on CPU only machines. I have tree hobby projects that I am working on exchangeably (currently focusing on the one that is not directly related to this particular topic) and coding is not my area of expertise, so I do not think I am in perfect position to make any contributions now. However, I fully agree with you regarding the Julia documentation about BLAS interactions and I still think that some blog post that summarize your and some other people findings on this subject would be an interesting read. :slight_smile: