# Import a module within that same module for parallelisation

**URL:** <https://discourse.julialang.org/t/import-a-module-within-that-same-module-for-parallelisation/27667>\
**Category:** General Usage\
**Tags:** parallel\
**Created:** [August 17, 2019, 10:21pm UTC](https://discourse.julialang.org/t/import-a-module-within-that-same-module-for-parallelisation/27667 "2019-08-17T22:21:45Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![s-baumann](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/s-baumann/32/5843_2.png) [@s-baumann](https://discourse.julialang.org/u/s-baumann)\
**Post date:** [August 17, 2019, 10:21pm UTC](https://discourse.julialang.org/t/import-a-module-within-that-same-module-for-parallelisation/27667/1 "2019-08-17T22:21:45Z")

</div>

I have a bunch of functions within a module. I want to make a function that uses various other functions within that module in parallel. The solution that works in the REPL is to `addprocs` for the number of cores I have and then use `@everywhere using ModuleName`. Within ModuleName however that does not work because there is an error "syntax: “toplevel” expression not a top level. A minimal working example is:

```julia
module ModuleName
    using Distributed
    function do_something(x)
        return x^2
    end
    function do_manythings(Y::Array, num_cores::Integer)
        addprocs(num_cores)
        @everywhere using ModuleName
        squares = @distributed (vcat) for y in Y
            do_something(y)
        end
        rmprocs(num_cores)
        return squares
    end
    export do_something, do_manythings
end

```

Potentially there is an issue that the compiler would like to see ModuleName in compiling this code but doesn’t have it yet (but I don’t really know how compilers work…).

What is the proper way to use the functions of a module within that same module? I would ideally like to do it so that a user of ModuleName specifies how many cores he/she has to use but does not have to know or do anything else to get it to run in parallel.

---

<div class="post-metadata">

**Author:** ![marius311](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marius311/32/3953_2.png) [@marius311](https://discourse.julialang.org/u/marius311)\
**Post date:** [August 17, 2019, 10:38pm UTC](https://discourse.julialang.org/t/import-a-module-within-that-same-module-for-parallelisation/27667/2 "2019-08-17T22:38:24Z")

</div>

You can use `@everywhere @eval Mod1 using Mod2` to evaluate `using Mod2` in `Mod1`’s namespace on the remote worker. Note `@eval ModuleName expr` can generally be used to evaluate anything into another module.

---

<div class="post-metadata">

**Author:** ![s-baumann](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/s-baumann/32/5843_2.png) [@s-baumann](https://discourse.julialang.org/u/s-baumann)\
**Post date:** [August 18, 2019, 2:42pm UTC](https://discourse.julialang.org/t/import-a-module-within-that-same-module-for-parallelisation/27667/3 "2019-08-18T14:42:26Z")

</div>

This seems to work. I replace `@everywhere using ModuleName` with `@everywhere @eval using ModuleName`. Just to check my understanding of what this is doing though:

- Is the @eval macro here acting to tell the compiler to ignore it and run the using command at runtime?
- Will this mean that each core will compile ModuleName itself or will each just use the already compiled binaries done on the main Julia thread.

---

<div class="post-metadata">

**Author:** ![marius311](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marius311/32/3953_2.png) [@marius311](https://discourse.julialang.org/u/marius311)\
**Post date:** [August 18, 2019, 7:05pm UTC](https://discourse.julialang.org/t/import-a-module-within-that-same-module-for-parallelisation/27667/4 "2019-08-18T19:05:30Z")

</div>

> [@s-baumann](#):
>
> - Is the @eval macro here acting to tell the compiler to ignore it and run the using command at runtime?

Not sure I understand exactly what you mean, but basically its the same thing as how you can’t just put `using Foo` inside the body of a function. But you can `@eval using Foo` inside a function to execute that statement in the toplevel scope of the current module.

> [@s-baumann](#):
>
> - Will this mean that each core will compile ModuleName itself or will each just use the already compiled binaries done on the main Julia thread.

As I just learned trying to answer this, looks like `@everywhere` is smart and will look through your expression, grab any `import` or `using` statements, and do those first on the master process, before sending the statement to workers. But what happens when it does `using Foo` on the master is special, and described [here](https://docs.julialang.org/en/v1/manual/parallel-computing/index.html#Code-Availability-and-Loading-Packages-1), especially the very last sentence. Basically you precompile `Foo` (if necessary) on the master process, load it on the master, then load it on the workers (which will use the compile cache). In your case you will then also execute `using Foo` on the workers, which will just bring `Foo` into scope and be instantaneous as `Foo` was already loaded.

---

<div class="post-metadata">

**Author:** ![s-baumann](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/s-baumann/32/5843_2.png) [@s-baumann](https://discourse.julialang.org/u/s-baumann)\
**Post date:** [August 18, 2019, 9:30pm UTC](https://discourse.julialang.org/t/import-a-module-within-that-same-module-for-parallelisation/27667/5 "2019-08-18T21:30:09Z")

</div>

ok. This makes alot of sense. Thanks for this.