Using local module with @everywhere

Specific Domains Julia at Scale
1

Hi, my goal is to within one script define a module only once in my local scope, and then read it to all my workers. However, I cannot figure out what to do. This is my MWE:

using Distributed

module myModule
export f
function f(x_in)
        return x_in^2.0
end
end

using .myModule
f(1.0) # This works as expected

addprocs(1)

@everywhere using .myModule # This throws error "myModule not defined"
@everywhere f(1.0)

If I put myModule in a separate myModule.jl, then the following works:

using Distributed

include("myModule.jl"); using .myModule
f(1.0)

addprocs(1)

@everywhere begin include("myModule.jl"); using .myModule end
@everywhere f(1.0)

This solution is OK, but I prefer to have everything in one script for reasons.

(The actual goal is to conveniently load the function f(x_in) without having to use the solution in Easy way to send custom function to distributed workers? - #2 by greg_plowman)

2

Seems you can define the module with @everywhere:

julia> using Distributed

julia> addprocs(1)
1-element Vector{Int64}:
 5

julia> @everywhere module SomeModuleName
         f() = 3
       end

julia> @everywhere using .SomeModuleName

julia> @everywhere println(SomeModuleName.f())
3
      From worker 5:    3

julia> 
julia>       From worker 3:     3
      From worker 2:    3
julia> 

julia>

But I have a feeling you’re going about things the wrong way.

1 Like
4

OK, so what I thought before was that variables created with @everywhere were only accessible “by the workers” and not in my local scope. But I see now that if I want to only once define the module or the function f(), I should do it once using @everywhere.

Thanks!

using Distributed
addprocs(1)

@everywhere module myModule
        export f
        function f(x_in)
                return x_in^2.0
        end
end
@everywhere using .myModule
f(1.0)
@everywhere f(1.0)
1 Like