# @everywhere inside a module

**URL:** <https://discourse.julialang.org/t/everywhere-inside-a-module/126135>\
**Category:** General Usage\
**Created:** [February 21, 2025, 2:55am UTC](https://discourse.julialang.org/t/everywhere-inside-a-module/126135 "2025-02-21T02:55:04Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![Bart\_van\_de\_Lint](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bart_van_de_lint/32/212161_2.png) [@Bart\_van\_de\_Lint](https://discourse.julialang.org/u/Bart_van_de_Lint)\
**Post date:** [February 21, 2025, 2:55am UTC](https://discourse.julialang.org/t/everywhere-inside-a-module/126135/1 "2025-02-21T02:55:04Z")

</div>

I am trying to parallelize `solve_alpha` from `Xfoil.jl`. The problem is: `Xfoil.jl` has global variables inside the package, so I cannot use `@threads` to run multiple `solve_alphas` at the same time. I have to use Distributed.jl with `@everywhere` to make multiple instances of Xfoil.jl. This works in a script, but it doesn’t work when I try to use `@everywhere` inside a module. And I want to be able to run the code inside a module.

The script that works:

```julia
using Distributed
using Xfoil

const procs = addprocs()

@everywhere begin
    using Xfoil
    function solve_alpha(x, y)
        Xfoil.set_coordinates(x, y)
        Xfoil.solve_alpha(0.0; mach=0.1)
        return nothing
    end
end

try
    x = collect(0:0.1:1)
    y = x .^ 2
    @show x y
    @sync @distributed for j in 1:10
        solve_alpha(x, y)
    end
catch e
    println(e)
finally
    println("removing processes")
    rmprocs(procs)
end

```

The module that doesn’t work:

```julia
module DistributedXfoil

using Distributed
using Xfoil

const procs = addprocs()

@everywhere begin
    using Xfoil
    function solve_alpha(x, y)
        Xfoil.set_coordinates(x, y)
        Xfoil.solve_alpha(0.0; mach=0.1)
        return nothing
    end
end

try
    x = collect(0:0.1:1)
    y = x .^ 2
    @show x y
    @sync @distributed for j in 1:10
        solve_alpha(x, y)
    end
catch e
    println(e)
finally
    println("removing processes")
    rmprocs(procs)
end

end

```

---

<div class="post-metadata">

**Author:** ![ufechner7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ufechner7/32/51363_2.png) [@ufechner7](https://discourse.julialang.org/u/ufechner7)\
**Post date:** [February 21, 2025, 9:25am UTC](https://discourse.julialang.org/t/everywhere-inside-a-module/126135/2 "2025-02-21T09:25:32Z")

</div>

> [@Bart\_van\_de\_Lint](#):
>
> The module that doesn’t work:

What is the error message, if any?

And what is the purpose of this line:

```julia
const procs = addprocs()

```

When shall addprocs() be called? I don’t see that you make use of `procs` in your code.

And I do not see the the line `using DistributedXfoil` in your code.

You must decide what you want:

- either to execute code when precompiling the module, or when loading the module
- or you have a separate init() function that you can call any time you want

---

<div class="post-metadata">

**Author:** ![Bart\_van\_de\_Lint](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bart_van_de_lint/32/212161_2.png) [@Bart\_van\_de\_Lint](https://discourse.julialang.org/u/Bart_van_de_Lint)\
**Post date:** [February 21, 2025, 3:33pm UTC](https://discourse.julialang.org/t/everywhere-inside-a-module/126135/3 "2025-02-21T15:33:31Z")

</div>

I found a solution. You can run a script from a module using `@eval Main include("polar_exec.jl")`

```julia
"src/DistributedXfoil.jl"
module DistributedXfoil

using Distributed
using Xfoil

export init

function init()
    @eval Main include("polar_exec.jl")
end

end

using .DistributedXfoil
init()

```

```julia
"src/polar_exec.jl"
using Distributed
using Xfoil

const procs = addprocs()

@everywhere begin
    using Xfoil
    function solve_alpha(x, y)
        Xfoil.set_coordinates(x, y)
        Xfoil.solve_alpha(0.0; mach=0.1)
        return nothing
    end
end

try
    x = collect(0:0.1:1)
    y = x .^ 2
    @show x y
    @sync @distributed for j in 1:10
        solve_alpha(x, y)
    end
catch e
    println(e)
finally
    println("removing processes")
    rmprocs(procs)
end

```

---

<div class="post-metadata">

**Author:** ![mbauman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mbauman/32/31082_2.png) [@mbauman](https://discourse.julialang.org/u/mbauman)\
**Post date:** [February 21, 2025, 3:52pm UTC](https://discourse.julialang.org/t/everywhere-inside-a-module/126135/4 "2025-02-21T15:52:31Z")

</div>

I mean, that might work in some situations, but it’s really not the solution you want.

You generally want modules to only define the machinery that they need. And then you create functions that can be called to run that machinery.

Detangling these things is a very good thing. Modules are generally setup such that they can be _defined_ at any time. And then they have functions that do the interesting work.

Or you can have scripts — external to the module — that run the functions defined by the module.

---

<div class="post-metadata">

**Author:** ![Bart\_van\_de\_Lint](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bart_van_de_lint/32/212161_2.png) [@Bart\_van\_de\_Lint](https://discourse.julialang.org/u/Bart_van_de_Lint)\
**Post date:** [February 21, 2025, 4:00pm UTC](https://discourse.julialang.org/t/everywhere-inside-a-module/126135/5 "2025-02-21T16:00:10Z")

</div>

But in this case, it is a way of automatically running the script from the Module. In the real module, I add checks to see if the script should run, and only run it if it needs to.

The script is still external to the model. And the user can run it themselves. But now the running of the script is automated. Maybe it would be better to do this automation in a bash script. But then the user has to run the bash script on startup instead of just running Julia.

---

<div class="post-metadata">

**Author:** ![abraemer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/abraemer/32/51403_2.png) [@abraemer](https://discourse.julialang.org/u/abraemer)\
**Post date:** [February 21, 2025, 4:57pm UTC](https://discourse.julialang.org/t/everywhere-inside-a-module/126135/6 "2025-02-21T16:57:35Z")

</div>

I think instead of wrapping this into a module, you could wrap it into a function and then call that like any other function.

You could then think about splitting the setup part (creating the workers and loading the dependencies) from the actual running of the computations.

---

<div class="post-metadata">

**Author:** ![Bart\_van\_de\_Lint](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bart_van_de_lint/32/212161_2.png) [@Bart\_van\_de\_Lint](https://discourse.julialang.org/u/Bart_van_de_Lint)\
**Post date:** [February 21, 2025, 5:00pm UTC](https://discourse.julialang.org/t/everywhere-inside-a-module/126135/7 "2025-02-21T17:00:35Z")

</div>

No, because `@everywhere` has to be called in the global scope. Wrapping in a function causes an error. Feel free to experiment with the solution I provided, but I wasn’t able to wrap it in a function.

---

<div class="post-metadata">

**Author:** ![abraemer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/abraemer/32/51403_2.png) [@abraemer](https://discourse.julialang.org/u/abraemer)\
**Post date:** [February 21, 2025, 5:03pm UTC](https://discourse.julialang.org/t/everywhere-inside-a-module/126135/8 "2025-02-21T17:03:15Z")

</div>

Ok I thought that could work. If not then you need to put a `@eval` in front of `@everywhere` and then it should work 🙂

---

<div class="post-metadata">

**Author:** ![Bart\_van\_de\_Lint](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bart_van_de_lint/32/212161_2.png) [@Bart\_van\_de\_Lint](https://discourse.julialang.org/u/Bart_van_de_Lint)\
**Post date:** [February 21, 2025, 5:21pm UTC](https://discourse.julialang.org/t/everywhere-inside-a-module/126135/9 "2025-02-21T17:21:19Z")

</div>

It doesn’t help. Using `@eval @everywhere` I get errors with undefined variables without names or undefined modules. The solution I provided is the only one that I got working.

```julia
module DistributedXfoil

using Distributed
using Xfoil

export init

function init()
    # @eval Main include("polar_exec.jl")

    procs = addprocs()
    @eval @everywhere begin
        using .DistributedXfoil, Xfoil
        function solve_alpha(x, y)
            Xfoil.set_coordinates(x, y)
            Xfoil.solve_alpha(0.0; mach=0.1)
            return nothing
        end
    end
        
    try
        x = collect(0:0.1:1)
        y = x .^ 2
        @show x y
        @sync @distributed for j in 1:10
            solve_alpha(x, y)
        end
    catch e
        println(e)
    finally
        println("removing processes")
        rmprocs(procs)
    end
end

end

using .DistributedXfoil
init()

```

Causes an error:

```julia
ERROR: LoadError: On worker 2:
UndefVarError: `DistributedXfoil` not defined
Stacktrace:
 [1] top-level scope
   @ ~/Code/MWE/mwe/polars.jl:13
 [2] eval
   @ ./boot.jl:385
 [3] #invokelatest#2
   @ ./essentials.jl:892
 [4] invokelatest
   @ ./essentials.jl:889
 [5] #114
   @ ~/.julia/juliaup/julia-1.10.8+0.x64.linux.gnu/share/julia/stdlib/v1.10/Distributed/src/process_messages.jl:303
 [6] run_work_thunk
   @ ~/.julia/juliaup/julia-1.10.8+0.x64.linux.gnu/share/julia/stdlib/v1.10/Distributed/src/process_messages.jl:70
 [7] run_work_thunk
   @ ~/.julia/juliaup/julia-1.10.8+0.x64.linux.gnu/share/julia/stdlib/v1.10/Distributed/src/process_messages.jl:79
 [8] #100
   @ ~/.julia/juliaup/julia-1.10.8+0.x64.linux.gnu/share/julia/stdlib/v1.10/Distributed/src/process_messages.jl:88

...and 15 more exceptions.

Stacktrace:
 [1] sync_end(c::Channel{Any})
   @ Base ./task.jl:455
 [2] macro expansion
   @ ./task.jl:487 [inlined]
 [3] remotecall_eval(m::Module, procs::Vector{Int64}, ex::Expr)
   @ Distributed ~/.julia/juliaup/julia-1.10.8+0.x64.linux.gnu/share/julia/stdlib/v1.10/Distributed/src/macros.jl:219
 [4] top-level scope
   @ ~/.julia/juliaup/julia-1.10.8+0.x64.linux.gnu/share/julia/stdlib/v1.10/Distributed/src/macros.jl:203
 [5] eval
   @ ./boot.jl:385 [inlined]
 [6] init()
   @ Main.DistributedXfoil ~/Code/MWE/mwe/polars.jl:12
 [7] top-level scope
   @ ~/Code/MWE/mwe/polars.jl:39
 [8] include(fname::String)
   @ Base.MainInclude ./client.jl:494
 [9] top-level scope
   @ REPL[1]:1
in expression starting at /home/bart/Code/MWE/mwe/polars.jl:39

```

---

<div class="post-metadata">

**Author:** ![abraemer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/abraemer/32/51403_2.png) [@abraemer](https://discourse.julialang.org/u/abraemer)\
**Post date:** [February 21, 2025, 6:13pm UTC](https://discourse.julialang.org/t/everywhere-inside-a-module/126135/10 "2025-02-21T18:13:34Z")

</div>

Not sure why you put the `.DistributedXfoil` there. Just remove that and it should fix the error you are seeing. Unfortunately, I cannot test right now, so I don’t know whether it truly works then.  
If you wanted to include the “local” package into each worker, you need a different setup to ensure that the workers can find it.

---

<div class="post-metadata">

**Author:** ![mbauman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mbauman/32/31082_2.png) [@mbauman](https://discourse.julialang.org/u/mbauman)\
**Post date:** [February 21, 2025, 6:19pm UTC](https://discourse.julialang.org/t/everywhere-inside-a-module/126135/11 "2025-02-21T18:19:28Z")

</div>

I guess the question is why do you want to put the `@everywhere` inside a module? The whole point of `@everywhere` is _to mess with `Main`_. Modules generally shouldn’t be messing with other modules like that.

I would suspect that the pain point you’re trying to solve is that you want users to just call `X.run_in_parallel()` and not need to worry about `addprocs` or worker state. But the control of workers and worker state is very much a global sort of thing.

---

<div class="post-metadata">

**Author:** ![Bart\_van\_de\_Lint](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bart_van_de_lint/32/212161_2.png) [@Bart\_van\_de\_Lint](https://discourse.julialang.org/u/Bart_van_de_Lint)\
**Post date:** [February 21, 2025, 10:01pm UTC](https://discourse.julialang.org/t/everywhere-inside-a-module/126135/12 "2025-02-21T22:01:24Z")

</div>

Good summary. I don’t want the user to think about this parallel stuff, the code needs to “just work” and “be fast”. @threads is not an option, as Xfoil.jl doesn’t support it (there are global variables in the Xfoil module, and each thread would need its own instance of the Xfoil module).