Can PyPlot generate plots from multiple threads?

General Usage Performance
,
1

I’m new to Julia but already impressed because Julia is 4x faster than GNU Octave at generating the approximately 6000 frames of a 3D plot animation (30 minutes versus 120 minutes) I am making. I am hoping to be able to use Julia’s Threads module to generate the animation frames even faster but my attempts at calling PyPlot from multiple threads have resulted in faults and error messages requesting that I submit a bug report. Below is a simple test program that recreates the fault. Note that test_parallel_print() works and test_parallel_plot() generates the fault.

# import Pkg; Pkg.add("PyPlot")
using Plots; pyplot();
using Printf;

function test_parallel_print()
	print("testing multithreaded print ...\n")
	
	nPrint = 10
	Threads.@threads for i = 1:nPrint
		println("i = $i on thread $(Threads.threadid())")
	end
end

function test_parallel_plot()
	print("testing multithreaded plot ...\n")
	
	nPlot = 3
	POV = Vector{Plots.Plot{Plots.PyPlotBackend}}(undef,nPlot) # Plot Object Vector
	Threads.@threads for i = 1:nPlot
		POV[i] = plot(rand(5))
		plot!(POV[i], ones(5)./i)
		fname = @sprintf("test_plot%05d.png", i);
		savefig(POV[i], fname)
	end
end
2

no. Many plotting libraries are stateful (for example, even if you use GR.jl, you still can’t do it). In the case of pyplot, you will also run into Python’s GIL, which means multi-threading will never work, maybe give multi-processing a shot

3

e.g.

(@v1.7) pkg> activate --temp
  Activating new project at `C:\Users\alexa\AppData\Local\Temp\jl_f27Tnv`

(jl_f27Tnv) pkg> add Distributed, Plots, PyPlot
[...]

julia> cd(dirname(Base.load_path()[1])) # change to project directory

julia> using Distributed

julia> addprocs(4, exeflags="--project=.")
4-element Vector{Int64}:
 2
 3
 4
 5

julia> @everywhere begin
           using Plots
           pyplot()
       end

julia> POV = pmap(1:4) do i
           plot(rand(5))
       end
4-element Vector{Plots.Plot{Plots.PyPlotBackend}}:
 Plot{Plots.PyPlotBackend() n=1}
 Plot{Plots.PyPlotBackend() n=1}
 Plot{Plots.PyPlotBackend() n=1}
 Plot{Plots.PyPlotBackend() n=1}
1 Like
4

Thanks, I’ll give multi-processing a shot in the next couple days.

5

Please pardon me for my slow response. I noticed some mistakes in my animation and I wanted to fix those first instead of trying to speed up the generation of an incorrect animation.

Your example was very helpful. I ended up using the following variation of it.

# import Pkg.add("Distributed")
using Distributed

addprocs(4, exeflags="--project=.")
@everywhere begin
	include("some1.jl")
end

pmap(1:6) do i
	cfse(0, i, 6)
end

I think it is neat (“neat” as in cool and also “neat” as in organizationally tidy helping reduce debugging and maintenance) that Julia allows the code for distributing the task among several CPU cores to be separate from the code that implements the task.

For those interested in timing when run on AMD Ryzen 7 4800H, GNU Octave took 120 minutes to generate the animation frames.
Julia 1.7.0 took 23.3 minutes.
Julia 1.7.0 took 13.1 minutes when Distributed among 4 CPU cores.
Julia 1.7.0 took 11.1 minutes when Distributed among 6 CPU cores. (a tenth of the GNU Octave time)

For those interested in the animation of a complex Fourier series tracing the letter ‘e’, the animation is at complex Fourier series tracing the letter 'e' - YouTube
and the full Julia source code is in the appendix of the (still evolving) essay at https://towardsdatascience.com/zenos-illustrative-example-bb371b99f25a?source=friends_link&sk=8a6f90c96adf5b4a28d223a4a0db7350

1 Like