Run multiple python instances with pycall in different threads

General Usage
,
2

I don’t have a working PyCall to hand, but with PythonCall the following works fine, and I’d expect PyCall to work just the same:

julia> @everywhere using PythonCall

julia> @time pmap(1:4) do i
           __main__, time = pyimport("__main__", "time")
           time.sleep(4 - i)
           __main__.foo = myid()
           time.sleep(i)
           i => __main__.foo
       end
  4.284934 seconds (268.97 k allocations: 14.611 MiB, 3.89% compilation time)
4-element Vector{Pair{Int64, Py}}:
 1 => 5
 2 => 2
 3 => 3
 4 => 4

In this code, each iteration sleeps for a total of 4 seconds, but sets __main__.foo to the worker ID at a different time, then returns __main__.foo at the end. The total run-time is just over 4 seconds so they are indeed in parallel. Since they return different numbers, they are running separate interpreters (if they were running the same interpreter, __main__.foo would be the same number in all workers.)

1 Like