In an effort to make my main calculation code transparent to the parallelized implement, I want to have a user-defined function, then run it in parts of the calculations, doing all the setting up of parallelized calculation “invisible”. I understand I need to make the user-defined function available across cores. As a MWE:
foo.jl
:
using Distributed
function p_call(func)
ps = addprocs(2)
r = pmap(x -> func(x), 1:4)
rmprocs(ps)
r
end
Use it as
include("foo.jl")
f(x) = x*im
println(p_call(f))
fails with
ERROR: LoadError: On worker 2:
UndefVarError: #f not defined
sort of expected; however, if the above is wrapped inside another function:
include("foo.jl")
function run_code()
f(x) = x*im
p_call(f)
end
println(run_code())
without explicitly making @everyhere f
, this script works. How to best understand this and what is the preferred way to take in a user-provided function f and use it in a parallelized calculation, without exposing the parallelize setup to the user code? Thanks!