Compilation behavior in the multi-threading context

General Usage
,
1

Oftentimes compilation takes some time before running the main code.

Given the following code

function fn(x, y, z)
    # some heavy work that needs to be compiled
end
spawn_fn(a...) = Threads.@spawn(fn(a...))
function multi_th(tks, a...)
    for i=eachindex(tks)
        tks[i] = spawn_fn(a...)
    end
    foreach(wait, tks)
end
const tks = Vector{Task}(undef, 100)
multi_th(tks, x, y, z)

, I wonder: when I run multi_th for the first time, will fn be compiled only once (and all threads waiting for it to be finished) or once per thread?

And another question is: can I compile multiple functions in different threads simultaneously to expedite my code execution?

function fn1(x, y)
    # some heavy work
end
function fn2(x, y)
    # some heavy work
end
Threads.@spawn(fn1(x, y))
fn2(x, y)

Are there some subtleties here? e.g. function dependency.

1 Like
2

Threads share working memory, thus compiled method code. Distributed computing is where you’d have to separately compile a method per process.

Now this is an interesting question. The JIT devdocs currently states:

Currently, only one thread at a time is permitted to enter the optimize-compile-link pipeline at a time, due to restrictions imposed by one of our linkers (RuntimeDyld). However, the JIT is designed to support concurrent optimization and compilation, and the linker restriction is expected to be lifted in the future when RuntimeDyld has been fully superseded on all platforms.

But not everything in the docs is up to date. The most recent talk I could find on this is from JuliaCon 2022 (Unlocking Julia’s LLVM JIT Compiler - Prem Chintalapudi). Since then, v1.10 (December 2023) has introduced parallel precompilation, but Chintalapudi’s master’s thesis distinguishes precompilation from JIT compilation, including how they could be parallelized at the time of writing (June 2023). Chapter 5 covers JIT concurrency along the lines of this topic (“multiple application threads…running compilation simultaneously”), and there’s a similar mention there:

In particular, LLVM’s old RuntimeDyld linker could not handle multithreaded linking, while LLVM’s new JITLink linker is not supported on some platforms that Julia supports, which precludes a redesign of parallel codegen. However, JITLink support is getting better, which could permit such a redesign in the future

That’s as far as I would read for now, so I’m still in the dark on whether current Julia releases parallelize any JIT compilation in multithreaded programs.

1 Like