Using PyCall on different threads crashes my Julia REPL

[Impressium]

Hi all

the following two lines crashes my Julia REPL.

julia> using PyCall

julia> [Threads.@spawn py"sum(i for i in range(10_000_000))" for i in 1:10]

Anyone with the same experience? Does anyone know why?

Cheers

[stevengj]

@Impressium

Anyone with the same experience? Does anyone know why?

Python isn’t thread safe.

(Realize that PyCall is calling a single instance of the standard Python interpreter libpython that shares memory with the Julia process. This Python interpreter does not support concurrent calls. PyCall is not launching separate python processes, which would be very slow (but thread-safe).)

[cjdoris]

If each thread acquired and released the Global Interpreter Lock (GIL) then it should work. I’m not sure if PyCall exposes that functionality. Even if that works, like multithreading from Python itself, this will only accelerate I/O bound tasks.

[stevengj]

@cjdoris

If each thread acquired and released the Global Interpreter Lock (GIL) then it should work. I’m not sure if PyCall exposes that functionality.

No, you’d have to ccall to PyEval_SaveThread and PyEval_RestoreThread yourself.