Correct calling order for modules and pmap?

General Usage
1

I have a module , my_module.jl, which includes functions that I would like to use with pmap. Currently, I have tried to execute this like:

addprocs(4)
include("my_module.jl")

@everywhere function pmap_f(x)
    return my_module.g(x)^2
end

xvals = [[x] for x in linspace(0,1)];

gvals = pmap(pmap_f, xvals)

rmprocs(workers())

But this just ends up generating an error that my_module not found in the current path. Can someone offer some guidance about the correct way to set up such a script?

2

Should not it be @everywhere include...?

Maybe add
@everywhere cd("path to my file")

3

I tried the callling sequence:

@everywhere cd("/Users/me/path_to_module/")
@everywhere include("my_module.jl")

and this ends up generating:

On worker 7:
UndefVarError: my_module not defined
4

This is the error returned by the two above line? It works in serial?
And @everywhere include("/Users/me/path_to_module/my_module.jl")?

5

Putting the fully qualified path in the @everywhere include command resolves this.

6

Is there a cleaner way to do this, without having to identify, beforehand, exactly where I am in the filesystem when calling the include?

7

I make all of my projects into a custom module, and softlink (ln -s in linux) the development directory into ~/.julia/v0.6 so it shows up as a directory alongside the packages that I’ve loaded from the official sources. Then for parallel I just need to @everywhere using MyModule .

There are subtle differences between using and include, and usually my needs are met with using. The other big benefit with using is that Revise.jl will keep it up to date, at least in the main worker.

1 Like