I am using Julia PyPlot to do postprocessing for simulation data. Here is the sample script for my parallel work:
using Distributed @everywhere using PyCall @everywhere matplotlib = pyimport("matplotlib") @everywhere matplotlib.use("Agg") @everywhere using PyPlot, Glob @everywhere function process(filename::String, dir::String=".") np = pyimport("numpy"); filehead, data, filelist = readdata(filename, dir=dir, verbose=false); # Postprocessing... plt.savefig("$(time).png") println("finished saving $(time)!") end # Define path and filenames dir = "."; filename = "y*.out"; # Find filenames # ...... # Processing @distributed for filename in filenames println("filename: $(filename)") process(filename, dir) end
In this way, I find all the filenames on one processor, distribute the names within workers, and do the plotting and saving on each worker using @distributed. Since there’s no dependency between processing different files, this seems to work.
I am wondering if there are better ways to do this, say, using @threads or channel? Any idea is appreciated!