Why can Julia spawn new processes but not new threads?

Internals & Design Internals
, , , ,
1

With Distributed.jl, you can create new worker processes easily with addprocs (each of which can have multiple threads), but once a Julia process starts you can’t add more threads to that process; why?

3 Likes
2

New foreign threads can be created already, to facilitate external code that needs dedicated threads.

For instance Base.Experimental.make_io_thread() does that julia/base/experimental.jl at 1ed2b98131cb7074cec5f556f222e8ad023285f9 · JuliaLang/julia · GitHub

julia> Threads.maxthreadid()
1

julia> Base.Experimental.make_io_thread()
true

julia> Threads.maxthreadid()
2

But adding/removing threads to/from the user threadpools has not been implemented yet.
I don’t believe there’s a technical limitation.

11 Likes