Using pycall from threads

New to Julia
,
1

I have a potentially stupid question:
Is it possible to use pycall / pyplot from several threads at the same time?

I would guess it is not, but I want to be sure… (I could imagine having a python instance per thread so that this could work?)

Anyways, this code (where I try to save existing figures in parallel)

using PyPlot
pygui(true)

figs=[]
for index in 1:10
    fig = plt.figure()
    plt.plot(rand(10))
    push!(figs,fig)
end

Threads.@threads for index in 1:length(figs)
    println("Thread #"*string(Threads.threadid())*" saves image "*string(index))
    fig = figs[index]
    fig.savefig("image_"*string(index)*".png")
end

throws an exception:

Fatal Python error: GC object already tracked

Thread 0x00001018 (most recent call first):

Is there a way around this?
It appears unrelated to file I/O, also happens if I try to manipulate a figure from threads.
(Of course, in my real example the saving of the figures takes at least 10s of seconds, otherwise I wouldnt care…)

2 Likes
2

There has been some discussion in the Python community about supporting “sub-interpreters”, see:

See below for a demo of loading multiple interpreters in the same process, using dlmopen to provide namespace isolation:

https://gist.github.com/dutc/eba9b2f7980f400f6287

A similar approach could be implemented in Julia. I don’t know if PyCall makes assumptions about global state on the Julia side, but some level of refactoring would likely be necessary.