# Making code and packages available to workers inside module

**URL:** <https://discourse.julialang.org/t/making-code-and-packages-available-to-workers-inside-module/56311>\
**Category:** General Usage\
**Tags:** parallel, module, distributed\
**Created:** [March 2, 2021, 8:35am UTC](https://discourse.julialang.org/t/making-code-and-packages-available-to-workers-inside-module/56311 "2021-03-02T08:35:14Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Nikos\_Gianniotis](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nikos_gianniotis/32/11487_2.png) [@Nikos\_Gianniotis](https://discourse.julialang.org/u/Nikos_Gianniotis)\
**Post date:** [March 2, 2021, 8:35am UTC](https://discourse.julialang.org/t/making-code-and-packages-available-to-workers-inside-module/56311/1 "2021-03-02T08:35:14Z")

</div>

Dear forum, my title explains to some extend what I am trying to achieve.

To make it hopefully clearer, I have written the following minimal code as an example.

First, I create an ordinary script (`scriptA.jl`) where the desired functionality is implemented:  
(1) there is a function called `forallworkers` that is made available to all available workers  
(2) there is a second function called `caller`, available only to the process handling the REPL (is this the right way of putting it?) which calls function `forallworkers`.

In my real problem, function `forallworkers` implements some tedius and lengthy calculations and function `caller` calls `forallworkers` multiple times in parallel using `pmap` while doing some extra calculations which are not relevant here. Hence, the user can comfortably call function `caller` while being oblivious to the fact that there are computations that run in parallel.

If I start julia 1.5.3 with `julia -O3 -p2`, load the script with `include("scriptA.jl")` below

```julia
# scriptA.jl
# starting julia 1.5.3 with: julia -O3 -p2

@everywhere using SpecialFunctions, Printf

@everywhere function forallworkers(x)
    @printf("Result from worker %d is %.3f\n", myid(), erf(x))
    return erf(x)
end

function caller(X)
    # some computation on X takes place here in the real problem, 
    # but is not relevant to this MWE
    pmap(forallworkers, X)
end

```

and call `caller(1:5)`, things will run as expected:

```julia
julia> caller(1:5)
      From worker 3:	Result from worker 3 is 0.995
      From worker 2:	Result from worker 2 is 0.843
      From worker 3:	Result from worker 3 is 1.000
      From worker 2:	Result from worker 2 is 1.000
      From worker 3:	Result from worker 3 is 1.000
5-element Array{Float64,1}:
 0.8427007929497149
 0.9953222650189527
 0.9999779095030014
 0.9999999845827421
 0.9999999999984626

```

Now, I would like to put this functionality in a module (with the future intention of creating a package). My first attempt was:

```julia
# scriptB.jl
# starting julia 1.5.3 with: julia -O3 -p2

module modMWE

    export caller

    @everywhere using SpecialFunctions, Printf

    @everywhere function forallworkers(x)
        @printf("Result from worker %d is %.3f\n", myid(), erf(x))
        return erf(x)

    end

    function caller(X)
        # some computation on X takes place here in the real problem, 
        # but is not relevant to this MWE
        pmap(forallworkers, X)
    end

end

```

However, if I load `scriptB.jl` immediately after starting julia with `include("scriptB.jl")`, I get the message

`"ERROR: LoadError: LoadError: UndefVarError: @everywhere not defined"`

which I found odd since I started julia with two processes using the flag `-p2`. Nevertheless, after inserting the line `using Distributed` the problem seems to go away in `scriptC.jl`:

```julia
# scriptC.jl
# starting julia 1.5.3 with: julia -O3 -p2

module modMWE

    export caller

    using Distributed # <------ newly added line

    @everywhere using SpecialFunctions, Printf

    @everywhere function forallworkers(x)
        @printf("Result from worker %d is %.3f\n", myid(), erf(x))
        return erf(x)
    end

    function caller(X)
        # some computation on X takes place here in the real problem, 
        # but is not relevant to this MWE
        pmap(forallworkers, X)
    end

end

```

Now I can successfully load the module with `include("scriptC.jl")`. However, if I now try and call function caller with `modMWE.caller(1:5)` this results in the following error:

```julia
julia> modMWE.caller(1:5)
ERROR: UndefVarError: forallworkers not defined
Stacktrace:
 [1] caller(::UnitRange{Int64}) at ./REPL[1]:15
 [2] top-level scope at REPL[2]:1

```

My naive impression is that function `forallworkers` should be available to all processes since its definition has been qualified with `@everywhere`, but obviously this is not the case.

My question therefore:

- How can I properly create a module that has functions that visible to all workers so that they can be run in parallel?
- I.e. in this particular example this specifically translates to, how do I make function `forallworkers` properly available to all workers and callable within function `caller`?

Many thanks!

* * *

EDIT: After solving the problem, I add here how the final version of the script (you can also find it below if you follow the discussion) that solves the problem.

```julia
# scriptE.jl
# starting julia 1.5.3 with: julia -O3 -p2

@everywhere module modMWE

    export caller

    using Distributed

    using SpecialFunctions, Printf

    function forallworkers(x)
        @printf("Result from worker %d is %.3f\n", myid(), erf(x))
        return erf(x)
    end

    function caller(X)
        # some computation on X takes place here in the real problem,
        # but is not relevant to this MWE
        pmap(forallworkers, X)
    end

end

```

---

<div class="post-metadata">

**Author:** ![marius311](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marius311/32/3953_2.png) [@marius311](https://discourse.julialang.org/u/marius311)\
**Post date:** [March 2, 2021, 8:52am UTC](https://discourse.julialang.org/t/making-code-and-packages-available-to-workers-inside-module/56311/2 "2021-03-02T08:52:32Z")

</div>

The key part of the [docs](https://docs.julialang.org/en/v1/stdlib/Distributed/#Distributed.@everywhere) for `@everywhere` to catch is “Execute an expression _under `Main`_”. So in scriptC when you have those `@everywhere`’s inside your module, that code isn’t evaluated in your module, its evaluated in `Main`, so you end up defining `Main.forallworkers` on all processes. You can see that even on the main process, `modMWE.forallworkers` isn’t defined.

The two things you can do are 1) remove the `@everywhere`s from inside the module and evaluate the _entire module definition_ on all workers, so `@everywhere module modMWE ...`. Then `modMWE.caller(1:5)` will work, or (better) 2) turn this into a bonafide package with a `Project.toml` and `src/` directory, etc…, then `using modMWE` automatically loads the module on all workers, without the need for `@everywhere`s anywhere.

Edit: I should also say, welcome, and thanks for posting a really clear MWE!

---

<div class="post-metadata">

**Author:** ![Nikos\_Gianniotis](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nikos_gianniotis/32/11487_2.png) [@Nikos\_Gianniotis](https://discourse.julialang.org/u/Nikos_Gianniotis)\
**Post date:** [March 2, 2021, 10:37am UTC](https://discourse.julialang.org/t/making-code-and-packages-available-to-workers-inside-module/56311/3 "2021-03-02T10:37:11Z")

</div>

@marius311 Thanks for the reply and the warm welcome!

I had forgotten that `@everywhere` should be executed under Main, so this is helpful already. Now I understand why `modMWE.forallworkers` is not visible to function `caller`, as you explain.

However, I am still not entirely clear how I should proceed despite your suggestions. In my attempt to follow your suggestion, I have now created a new script called `scriptD.jl` which reads as follows:

```julia
# scriptD.jl
# starting julia 1.5.3 with: julia -O3 -p2

@everywhere module modMWE

    export caller

    using Distributed

    using SpecialFunctions, Printf

    function forallworkers(x)
        @printf("Result from worker %d is %.3f\n", myid(), erf(x))
        return erf(x)
    end

    function caller(X)
        # some computation on X takes place here in the real problem, 
        # but is not relevant to this MWE
        map(forallworkers, X)
    end

end

```

I load the script again with `include("scriptD.jl")`. If I now call my function `caller`, I get the following output on the REPL:

```julia
julia> modMWE.caller(1:5)
Result from worker 1 is 0.843
Result from worker 1 is 0.995
Result from worker 1 is 1.000
Result from worker 1 is 1.000
Result from worker 1 is 1.000
5-element Array{Float64,1}:
 0.8427007929497149
 0.9953222650189527
 0.9999779095030014
 0.9999999845827421
 0.9999999999984626

```

Repeating the call, I seem to always get the exact same output: worker 1 always shows up, but other workers never do, despite Julia informing that there are indeed two workers available:

```julia
julia> nworkers()
2

```

What I understand, is that even though function `caller` must be available to all workers, me calling it from the REPL results in only worker 1 doing the work because it is the worker that is “responsible” for the REPL.

If I explicitly call things in parallel, like:

```julia
pmap(modMWE.caller, 1:5)
      From worker 2:	Result from worker 2 is 0.843
      From worker 3:	Result from worker 3 is 0.995
      From worker 3:	Result from worker 3 is 1.000
      From worker 2:	Result from worker 2 is 1.000
      From worker 3:	Result from worker 3 is 1.000
5-element Array{Float64,1}:
 0.8427007929497149
 0.9953222650189527
 0.9999779095030014
 0.9999999845827421
 0.9999999999984626

```

this works fine.

But: how can I make function `caller` internally distribute the work to other workers without me having to call things explicitly in parallel like above with e.g. `pmap(caller, 1:5)`.

Many thanks in advance!

---

<div class="post-metadata">

**Author:** ![marius311](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marius311/32/3953_2.png) [@marius311](https://discourse.julialang.org/u/marius311)\
**Post date:** [March 2, 2021, 11:37am UTC](https://discourse.julialang.org/t/making-code-and-packages-available-to-workers-inside-module/56311/4 "2021-03-02T11:37:14Z")

</div>

Looks like you switched `caller` from using `pmap` to using `map`, just keep it as `pmap`.

---

<div class="post-metadata">

**Author:** ![Nikos\_Gianniotis](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nikos_gianniotis/32/11487_2.png) [@Nikos\_Gianniotis](https://discourse.julialang.org/u/Nikos_Gianniotis)\
**Post date:** [March 2, 2021, 11:49am UTC](https://discourse.julialang.org/t/making-code-and-packages-available-to-workers-inside-module/56311/5 "2021-03-02T11:49:43Z")

</div>

@marius311 Sorry for this typo!

Below the final script that works indeed as expected:

```julia
# scriptE.jl
# starting julia 1.5.3 with: julia -O3 -p2

@everywhere module modMWE

    export caller

    using Distributed

    using SpecialFunctions, Printf

    function forallworkers(x)
        @printf("Result from worker %d is %.3f\n", myid(), erf(x))
        return erf(x)
    end

    function caller(X)
        # some computation on X takes place here in the real problem,
        # but is not relevant to this MWE
        pmap(forallworkers, X)
    end

end

```

Load with `include("scriptE.jl")` and call:

```julia
julia> modMWE.caller(1:5)
      From worker 2:	Result from worker 2 is 0.843
      From worker 3:	Result from worker 3 is 0.995
      From worker 3:	Result from worker 3 is 1.000
      From worker 2:	Result from worker 2 is 1.000
      From worker 3:	Result from worker 3 is 1.000
5-element Array{Float64,1}:
 0.8427007929497149
 0.9953222650189527
 0.9999779095030014
 0.9999999845827421
 0.9999999999984626

```

Let me see if I understand why the above works: qualifying the module definition with `@everywhere` makes the module available to all workers. As the module is available to all workers, so are its functions. It is as simple as that, right?