# Packages and workers
**URL:** https://discourse.julialang.org/t/packages-and-workers/14072
**Category:** General Usage
**Tags:** pkg, package-manager, distributed
**Created:** [August 26, 2018, 9:13am UTC](https://discourse.julialang.org/t/packages-and-workers/14072 "2018-08-26T09:13:59Z")
**Posts on this page:** 16
**Page:** 1
### Author:  [@teored90](https://discourse.julialang.org/u/teored90)
#### Post date: [August 26, 2018, 9:13am UTC](https://discourse.julialang.org/t/packages-and-workers/14072/1 "2018-08-26T09:13:59Z")
I don’t understand how to import a package that I generated on multiple workers on Julia v1.0.
Steps to reproduce:
1. Follow [the documentation](https://docs.julialang.org/en/stable/stdlib/Pkg/index.html#Creating-your-own-packages-1) to create the `HelloWorld` package.
2. Start Julia with `-p 2`.
3. Activate the environment with `] activate .`
4. `import HelloWorld`.
I get
```julia
ERROR: On worker 2:
ArgumentError: Package HelloWorld not found in current path:
- Run `Pkg.add("HelloWorld")` to install the HelloWorld package.
require at ./loading.jl:817
eval at ./boot.jl:319
#116 at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/Distributed/src/process_messages.jl:276
run_work_thunk at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/Distributed/src/process_messages.jl:56
run_work_thunk at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/Distributed/src/process_messages.jl:65
#102 at ./task.jl:259
#remotecall_wait#154(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::Function, ::Distributed.Worker, ::Module, ::Vararg{Any,N} where N) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/Distributed/src/remotecall.jl:407
remotecall_wait(::Function, ::Distributed.Worker, ::Module, ::Vararg{Any,N} where N) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/Distributed/src/remotecall.jl:398
#remotecall_wait#157(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::Function, ::Int64, ::Module, ::Vararg{Any,N} where N) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/Distributed/src/remotecall.jl:419
remotecall_wait(::Function, ::Int64, ::Module, ::Vararg{Any,N} where N) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/Distributed/src/remotecall.jl:419
(::getfield(Distributed, Symbol("##163#165")){Module,Expr})() at ./task.jl:259
...and 2 more exception(s).
Stacktrace:
[1] sync_end(::Array{Any,1}) at ./task.jl:226
[2] remotecall_eval(::Module, ::Array{Int64,1}, ::Expr) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/Distributed/src/macros.jl:207
[3] top-level scope at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/Distributed/src/macros.jl:190
```
Same result if I use `@everywhere`. If I start Julia with only one process the import is fine.
What am I missing?
---
### Author:  [@tamasgal](https://discourse.julialang.org/u/tamasgal)
#### Post date: [August 26, 2018, 9:20am UTC](https://discourse.julialang.org/t/packages-and-workers/14072/2 "2018-08-26T09:20:59Z")
I am still a total Pkg3 newbie but I got rid of your error (which I reproduced) by doing
```julia
using Pkg
Pkg.instantiate()
```
After that `using HelloWorld` works.
---
### Author:  [@teored90](https://discourse.julialang.org/u/teored90)
#### Post date: [August 26, 2018, 10:32am UTC](https://discourse.julialang.org/t/packages-and-workers/14072/3 "2018-08-26T10:32:03Z")
It doesn’t solve the problem for me. I still get a similar error
```julia
**ERROR:** On worker 2:
ArgumentError: Package HelloWorld [cf8bea5c-a91a-11e8-2ba1-4f0c15a7916a] is required but does not seem to be installed:
- Run `Pkg.instantiate()` to install all recorded dependencies.
```
When do you call `Pkg.instantiate()`? After having activated the environment?
---
### Author:  [@teored90](https://discourse.julialang.org/u/teored90)
#### Post date: [August 26, 2018, 10:46am UTC](https://discourse.julialang.org/t/packages-and-workers/14072/4 "2018-08-26T10:46:56Z")
[This issue](https://github.com/JuliaLang/julia/issues/28781) explains the problem. `activate .` is only executed on the master process, so I need a
```julia
using Distributed
using HelloWorld # works
addprocs(3)
@everywhere
using Pkg; Pkg.activate(".") # required
using HelloWorld
end
```
to get it loaded correctly.
I’d say its a pretty convoluted way of loading a package. Also, there should be some sort of warning or you lose a morning to figure this out.
---
### Author:  [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)
#### Post date: [August 27, 2018, 7:14am UTC](https://discourse.julialang.org/t/packages-and-workers/14072/5 "2018-08-27T07:14:39Z")
The [documentation](https://docs.julialang.org/en/v1/manual/parallel-computing/) does a pretty good job at explaining this.
> In order to refer to `MyType` across all processes, `DummyModule.jl` needs to be loaded on every process. Calling `include("DummyModule.jl")` loads it only on a single process. To load it on every process, use the [`@everywhere`](https://docs.julialang.org/en/v1/stdlib/Distributed/#Distributed.@everywhere) macro (starting Julia with `julia -p 2` ):
---
### Author:  [@jtackm](https://discourse.julialang.org/u/jtackm)
#### Post date: [August 27, 2018, 8:34am UTC](https://discourse.julialang.org/t/packages-and-workers/14072/6 "2018-08-27T08:34:07Z")
Still, assuming that users would like to use the activated environment (and loaded packages) across workers seems like a reasonable default to me. Are there any technical/usability reasons why this is not the case?
---
### Author:  [@teored90](https://discourse.julialang.org/u/teored90)
#### Post date: [August 27, 2018, 8:55am UTC](https://discourse.julialang.org/t/packages-and-workers/14072/7 "2018-08-27T08:55:10Z")
The documentation fails at mentioning the need to activate the environment with @everywhere. As I wrote in the OP the @everywhere import fails
---
### Author:  [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)
#### Post date: [August 27, 2018, 8:58am UTC](https://discourse.julialang.org/t/packages-and-workers/14072/8 "2018-08-27T08:58:28Z")
If you activate the environment and restart julia, do you still have to activate it to import on all workers? Since you start julia with workers in a state where the environment is not fully set up, maybe code loading on the workers is affected. Once the environment has been set up, new workers started will perhaps have this same environment?
---
### Author:  [@teored90](https://discourse.julialang.org/u/teored90)
#### Post date: [August 27, 2018, 3:00pm UTC](https://discourse.julialang.org/t/packages-and-workers/14072/9 "2018-08-27T15:00:29Z")
Yes, because Julia starts in the `v1.0` environment by default. I also tried activating the environment first and then launching the worker processes with `addprocs()`, but I still get a `ArgumentError: Package HelloWorld not found in current path:` from Worker 2.
---
### Author:  [@ksmcreynolds](https://discourse.julialang.org/u/ksmcreynolds)
#### Post date: [August 27, 2018, 3:19pm UTC](https://discourse.julialang.org/t/packages-and-workers/14072/10 "2018-08-27T15:19:10Z")
I solved this issue in my case by passing the `--project` flag to `addprocs`. This will load the project files for the current directory.
Start Julia using `julia --project`.
```julia
using Distributed
addprocs(2; exeflags="--project")
```
You can also pass the path to the desired environment using `--project=/path/to/env`. As long as the path in `addprocs` matches the one used to start Julia it should work fine.
---
### Author:  [@teored90](https://discourse.julialang.org/u/teored90)
#### Post date: [August 27, 2018, 4:01pm UTC](https://discourse.julialang.org/t/packages-and-workers/14072/11 "2018-08-27T16:01:29Z")
Thanks, this is an interesting workaround.
---
### Author:  [@lcosta](https://discourse.julialang.org/u/lcosta)
#### Post date: [October 16, 2018, 6:52pm UTC](https://discourse.julialang.org/t/packages-and-workers/14072/12 "2018-10-16T18:52:24Z")
I seem to be getting the same issue on v1.0 but the Pkg.activate fix doesn’t work. As a test, I’m running…
using Distributed
addprocs(2)
@everywhere push!(LOAD\_PATH, “.”)
@everywhere using Pkg
@everywhere Pkg.activate(“.”)
@everywhere using M
…where module M only contains the line println(“Loaded M on $(myid())”). This returns the error
LoadError: On worker 2: Failed to precompile M [top-level]
As with teored90’s issue, it works just fine in serial. When I attempt to add a package via _@everywhere using_ (was testing with “Interpolations”) rather than a user defined module it also fails, only with a directory error instead of a precompile one (this also works in serial). The _addprocs(2; exeflags=“–project”)_ fix mentioned by ksmcreynolds didn’t seem to have any effect. Has anyone seen / figured out a way around this problem? Thanks!
---
### Author:  [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)
#### Post date: [December 20, 2018, 9:18pm UTC](https://discourse.julialang.org/t/packages-and-workers/14072/13 "2018-12-20T21:18:23Z")
I just ran into this issue as well. I find the current default unintuitive, but perhaps there are good reasons for it.
---
### Author:  [@dave.f.kleinschmidt](https://discourse.julialang.org/u/dave.f.kleinschmidt)
#### Post date: [January 7, 2019, 10:00pm UTC](https://discourse.julialang.org/t/packages-and-workers/14072/14 "2019-01-07T22:00:13Z")
There’s an open issue about this here: [Workers should inherit Pkg environment · Issue #28781 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/issues/28781)
The problem AFAIU is that there’s no way to guarantee that the paths for `dev`ed packages are portable across machines (which can happen e.g. on a large cluster). Here’s @StefanKarpinski [(link)](https://github.com/JuliaLang/julia/issues/28781#issuecomment-415082809)
> This requires some careful thinking about how best to do this. One possibility is to send a manifest from the master node to the workers and insist that the manifest be usable on all nodes. That’s fine for non-dev packages and even for dev packages with relative paths but dev packages with absolute paths may be a bit of an issue. We could rewrite dev paths to use a commit instead and then send that over; of course, it requires that the workers know about the tree hash that’s being used…
The recommended work around is suggested by @simonbyrne:
> Another work around is to use the `JULIA_PROJECT` environment variable instead of `--project` , as that will be passed to subprocesses.
---
### Author:  [@jmcastro2109](https://discourse.julialang.org/u/jmcastro2109)
#### Post date: [August 22, 2019, 2:20pm UTC](https://discourse.julialang.org/t/packages-and-workers/14072/15 "2019-08-22T14:20:01Z")
Is there any final solution in this issue? I seem to have this problem and the proposed solutions don’t seem to work.
I started a topic with my case here: [Problems parallelizing declaring used packages - #9 by jmcastro2109](https://discourse.julialang.org/t/problems-parallelizing-declaring-used-packages/27728/9)
---
### Author:  [@Ahmed\_Salih](https://discourse.julialang.org/u/Ahmed_Salih)
#### Post date: [September 2, 2022, 10:07pm UTC](https://discourse.julialang.org/t/packages-and-workers/14072/16 "2022-09-02T22:07:25Z")
> [@ksmcreynolds](#):
>
> `addprocs(2; exeflags="--project")`
On Julia 1.8 this seemed to work for me too. I also added some @everywhere in main script, but not sure if needed or not. Thank you for this comment saved me a lot of hair pulling.
Kind regards