# Precompilation, environments, and \`Distributed\` computing

**URL:** <https://discourse.julialang.org/t/precompilation-environments-and-distributed-computing/22431>\
**Category:** General Usage\
**Tags:** question, package, pkg, distributed\
**Created:** [March 27, 2019, 9:39pm UTC](https://discourse.julialang.org/t/precompilation-environments-and-distributed-computing/22431 "2019-03-27T21:39:26Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![jacob-roth](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jacob-roth/32/1862_2.png) [@jacob-roth](https://discourse.julialang.org/u/jacob-roth)\
**Post date:** [March 27, 2019, 9:39pm UTC](https://discourse.julialang.org/t/precompilation-environments-and-distributed-computing/22431/1 "2019-03-27T21:39:26Z")

</div>

I’m using `Distributed.jl` to run simulations in parallel from a Jupyter Notebook, and my simulations are performed using a local repo `SimModels` (I have not yet made it a module) which `pmap`s various models/simulations to each processor from `Distributed`. I’m encountering similar issues to [this](https://discourse.julialang.org/t/precompilation-error-using-hpc/17094) question and am hoping for some guidance on how to setup my workflow.

Right now, I’m:

1. Setting Distributed environment by: `using Distributed; addprocs(20)`
2. Setting up the environment:  
`@everywhere using Pkg; @everywhere Pkg.activate("SimEnv"); @everywhere Pkg.instantiate()`
3. Including my code everywhere: `@everywhere include("..path-to-code/SimModels.jl")`

But when doing this, I am getting errors from the third step like

```julia
From worker 14:	│ exception = Required dependency Compat [34da2185-b29b-5c13-b0c7-acf172513d20] failed to load from a cache file.

```

Ideally I would be able to run a `setup.jl` script on a single processor which sets the whole `Distributed` environment, precompiles the necessary packages once, and then loads my local code on each processor in a way that uses the precompiled code.

Am I thinking about this correctly, or how might I go about this in a better way? Thanks!

---

<div class="post-metadata">

**Author:** ![daharn](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/daharn/32/27088_2.png) [@daharn](https://discourse.julialang.org/u/daharn)\
**Post date:** [July 13, 2021, 1:02pm UTC](https://discourse.julialang.org/t/precompilation-environments-and-distributed-computing/22431/2 "2021-07-13T13:02:38Z")

</div>

Did you find a working solution for this? I have similar problems, in such a way as that each worker tends to precompile all packages of the environment. This fails, presumably because they are all trying to do it at the same time.

Sometimes I get it to work, though I haven’t found out yet exactly how.