Is it possible to load a jl file on a worker as they start?
-L flag does not seems to work.
using Distributed open("worker_startup.jl", "w") do io print(io, """ using Distributed id = Distributed.myid() println("worker \$(id) started !!") """) close(io) end addprocs(1; exeflags="-L worker_startup.jl") @everywhere @show id
The point of this is that if I have an
@async task that starts workers on a cluster, I can start a
pmap call without waiting for all the workers to start.
Hoping that newly added workers will load their startup file, and then join the worker pool of the already running
task = @async for k=1:10 sleep(1) addprocs(1; exeflags="-L worker_startup.jl") end pmap(q->(sleep(1); rand()+myid()), 1:10)