@everywhere takes a very long time when using a cluster

[pfarndt]

I raised a similar (probably the same) issue here:

Module loading on workers

Through some trials and errors I realized that modules, which are loaded with using, are also loaded on the already existing workers. Therefore the following code runs without an error: using Distributed addprocs(2) using PyPlot pmap(workers()) do w w, myid(), PyPlot.version end I find this behavior quite inconvenient because in a situation with many workers (to generate some data) connected to one jupyter notebook kernel (to analyze this data) I rather prefer not to load a plotting packa…

I am still not satisfied with the current situation but now know how to avoid the greatest pitfalls, e.g. first load all the packages you need on the master, then add workers, then load packages you definitely need at workers.