# Use package not in current environment for testing/development

**URL:** <https://discourse.julialang.org/t/use-package-not-in-current-environment-for-testing-development/70424>\
**Category:** General Usage\
**Created:** [October 26, 2021, 6:23pm UTC](https://discourse.julialang.org/t/use-package-not-in-current-environment-for-testing-development/70424 "2021-10-26T18:23:33Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [October 26, 2021, 6:23pm UTC](https://discourse.julialang.org/t/use-package-not-in-current-environment-for-testing-development/70424/1 "2021-10-26T18:23:33Z")

</div>

There is one part of the development workflow that I may not be doing quite right.

When I am developing a package, the environment that package is generally the one activated.

Frequently, I have to use, for testing, some other packages which are not part of the package environment, meaning they are not dependencies of the package in question.

If the package is installed in the “main” environment, that is fine, because I can just use them. However, that leads to the progressive cluttering of the “main” environment.

Ideally, I would like to use packages without having to `add` them to the current environment (I don’t want them to be added to `Project.toml`).

One option is to have a third environment only for the development of the package, including test packages and `dev`ing the project in question. However, this is inconvenient because the corresponding `Project.toml` file cannot be placed in the same directory of the package itself.

Am I missing some other alternative? How do you use packages for testing and development which are not part of the project?

---

<div class="post-metadata">

**Author:** ![Egwene\_al\_Vere](https://avatars.discourse-cdn.com/v4/letter/e/df788c/32.png) [@Egwene\_al\_Vere](https://discourse.julialang.org/u/Egwene_al_Vere)\
**Post date:** [October 26, 2021, 6:35pm UTC](https://discourse.julialang.org/t/use-package-not-in-current-environment-for-testing-development/70424/2 "2021-10-26T18:35:52Z")

</div>

If the package is in the main env, should be available regardless? [package - Julia - If I activate a new environment, why can I still load modules from my general environment? - Stack Overflow](https://stackoverflow.com/questions/65858895/julia-if-i-activate-a-new-environment-why-can-i-still-load-modules-from-my-ge)

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [October 26, 2021, 6:36pm UTC](https://discourse.julialang.org/t/use-package-not-in-current-environment-for-testing-development/70424/3 "2021-10-26T18:36:02Z")

</div>

“Dev packages”: for those I use environment stacking, i.e. I have dev packages like Revies, BenchmarkTools etc. in the default / global environment. Hence they can always be loaded (unless I mangle with the `LOAD_PATH`).

“Test packages”: Well, I put them under extra / targets test. If you want you can use [TestEnv.jl](https://github.com/JuliaTesting/TestEnv.jl)

> [@lmiq](#):
>
> One option is to have a third environment only for the development of the package, including test packages and `dev` ing the project in question. However, this is inconvenient because the corresponding `Project.toml` file cannot be placed in the same directory of the package itself.

I often have a `scripts` or `examples` folder in my package repo in which I `] dev ..` or `push!` the main package into the `LOAD_PATH`.

Having said all of the above, I agree that we could still need a bit more tooling / convenience functions for testing / developing. For example, we currently don’t have a nice way to add packages to the test environment (i.e. the test target section).

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [October 26, 2021, 7:07pm UTC](https://discourse.julialang.org/t/use-package-not-in-current-environment-for-testing-development/70424/4 "2021-10-26T19:07:43Z")

</div>

> [@carstenbauer](#):
>
> Test packages”: Well, I put them under extra / targets test.

This has to be done by hand? Meaning, by editing the `Project.toml` file and moving it there after adding it?

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [October 26, 2021, 7:08pm UTC](https://discourse.julialang.org/t/use-package-not-in-current-environment-for-testing-development/70424/5 "2021-10-26T19:08:25Z")

</div>

> [@Egwene\_al\_Vere](#):
>
> If the package is in the main env, should be available regardless?

Yes, they are, that is one option, but the main environment become cluttered very rapidly.

---

<div class="post-metadata">

**Author:** ![mauro3](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mauro3/32/292_2.png) [@mauro3](https://discourse.julialang.org/u/mauro3)\
**Post date:** [October 26, 2021, 7:12pm UTC](https://discourse.julialang.org/t/use-package-not-in-current-environment-for-testing-development/70424/6 "2021-10-26T19:12:20Z")

</div>

I think this is [https://github.com/JuliaLang/Pkg.jl/issues/1233](https://github.com/JuliaLang/Pkg.jl/issues/1233), which has no solution yet…

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [October 26, 2021, 8:43pm UTC](https://discourse.julialang.org/t/use-package-not-in-current-environment-for-testing-development/70424/7 "2021-10-26T20:43:42Z")

</div>

> [@lmiq](#):
>
> This has to be done by hand? Meaning, by editing the `Project.toml` file and moving it there after adding it?

Yes

---

<div class="post-metadata">

**Author:** ![MA\_Laforge](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ma_laforge/32/385_2.png) [@MA\_Laforge](https://discourse.julialang.org/u/MA_Laforge)\
**Post date:** [October 27, 2021, 3:05am UTC](https://discourse.julialang.org/t/use-package-not-in-current-environment-for-testing-development/70424/8 "2021-10-27T03:05:50Z")

</div>

### Environments: My understanding

Julia appears to have been developed more as an “analysis” language than a language to develop end-user “applications”.

In other words, the intent is for users to launch Julia in a given “environment” that is appropriate for solving certain types of problems. Note that this environment is typically “interactive” (ex: REPL, Jupyter, Pluto, …).

### Suggested Julia usage flow

I used to launch Julia from any old directory where my data and “scripts” were found.

Lately, I started ensuring these directories are actually “proper Julia environments”. The following is an example of how I might organize said environments:

```julia-auto
/path/to/JuliaRunEnvironments
├─ GrammarOfGraphics
| └─ Project.toml
├─ CircuitDesign
| └─ Project.toml
└─ SatelliteImagery
   └─ Project.toml

```

What do I consider to be a “proper Julia environment”?

- Quite simply, an environment is a folder that includes a “Project.toml” file appropriate for the analysis I am performing.
- In other words: an environment should include any/all the packages you might need for a particular analysis task.

Applying this methodology, you can mitigate conflicts where packages required in two different solution spaces might otherwise force certain dependencies to be downgraded. This is not possible if you are working in the default `(@v1.6)` environment.

Regarding suggestions wrt the environment stack:

- Personally, I rarely push multiple `Project.toml`-based environments onto my environment stack.
- On startup of a new Julia session, I might make available a local library of custom packages using `push!(Base.LOAD_PATH, "/path/to/MyPackageRepo")` — but that’s about it.
- I prefer to keep my default/base `(@v1.6)` environment relatively clean/minimal to avoid issues.

### Sample set of run environments

For illustration purposes, a developer might want the following set of what I would call “run” environments:

```julia-auto
/path/to/JuliaRunEnvironments
├─ GrammarOfGraphics
| ├─ Project.toml
| ├─ ArtProjects
| | └─ starrynight_on_julia.jl
| | └─ monalisa_smiling.jl
| | └─ monalisa_frowning.jl
| └─ TestPatterns
| └─ nautilus_drawings.jl
| └─ fractal_trees.jl
├─ CircuitDesign
| ├─ Project.toml
| ├─ DeltaSigma
| | └─ 1_bit_dac.jl
| | └─ 8_bit_adc.jl
| └─ Filters
| └─ butterworth.jl
| └─ chebyshev.jl
└─ SatelliteImagery
   ├─ Project.toml
   ├─ MarsRover
   | └─ detect_iron_concentration.jl
   └─ Hubble
      └─ compensate_for_gravity_lensing.jl

```

Seems to me like an ideal strategy to house domain-specific scripts.

### So when do I `]dev` a package under development, exactly?

Typically only when I want to add/remove dependencies from said package.

Otherwise, I tend to create environments like:

```julia-auto
/path/to/JuliaDevEnvironments
├─ MyPackageDev
| ├─ Project.toml
| └─ ...
└─ ...

```

which would, of course, include an appropriate `Project.toml` file.

These environments are very light weight (basically one file). You just need to take a few extra steps to create them.

Then, when I wish to _ **develop** _ said package, I simply:

```julia-auto
$ cd /path/to/JuliaDevEnvironments/MyPackageDev
$ julia
julia> ]
(@v1.6) pkg> activate .
(MyPackageDev) pkg>
julia> #Ready to start!

```

---

<div class="post-metadata">

**Author:** ![GunnarFarneback](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gunnarfarneback/32/1827_2.png) [@GunnarFarneback](https://discourse.julialang.org/u/GunnarFarneback)\
**Post date:** [October 27, 2021, 5:46am UTC](https://discourse.julialang.org/t/use-package-not-in-current-environment-for-testing-development/70424/9 "2021-10-27T05:46:20Z")

</div>

> [@MA\_Laforge](#):
>
> Then, when I wish to _ **develop** _ said package, I simply:
> 
> ```julia
> $ cd /path/to/JuliaDevEnvironments/MyPackageDev
> $ julia
> julia> ]
> (@v1.6) pkg> activate .
> (MyPackageDev) pkg>
> julia> #Ready to start!
> 
> ```

This is what I do too, but I type it as

```julia
$ cd /path/to/JuliaDevEnvironments/MyPackageDev
$ julia --project
julia> #Ready to start!

```

---

<div class="post-metadata">

**Author:** ![gustaphe](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gustaphe/32/18174_2.png) [@gustaphe](https://discourse.julialang.org/u/gustaphe)\
**Post date:** [October 27, 2021, 6:02am UTC](https://discourse.julialang.org/t/use-package-not-in-current-environment-for-testing-development/70424/10 "2021-10-27T06:02:32Z")

</div>

One option is

```nohighlight
cd ~/.julia/dev/MyPackage
mkdir development
echo development/ >> .git/info/exclude
cd development
julia
]activate .
]dev MyPackage

```

Now you have a development environment for your package which doesn’t clutter the repo. Here you can put whatever scripts you want, or just repl around.

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [October 27, 2021, 12:16pm UTC](https://discourse.julialang.org/t/use-package-not-in-current-environment-for-testing-development/70424/11 "2021-10-27T12:16:04Z")

</div>

I think this does not really work, it conflicts with the environment of the package, for example when running tests: [https://github.com/JuliaLang/Pkg.jl/issues/1585](https://github.com/JuliaLang/Pkg.jl/issues/1585)

(I had this same error trying to use an environment in a subfolder of my project).

---

<div class="post-metadata">

**Author:** ![RogerP](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rogerp/32/17496_2.png) [@RogerP](https://discourse.julialang.org/u/RogerP)\
**Post date:** [October 30, 2021, 1:53pm UTC](https://discourse.julialang.org/t/use-package-not-in-current-environment-for-testing-development/70424/12 "2021-10-30T13:53:18Z")

</div>

@MA_Laforge I am also having brain muddle over this. I am developing a modular real-time control system application that has several packages (e.g. for telemetry, automated model-building, database interfaces, also a dynamic simulator, FMI/FMU blah blah etc. even a GUI). I am making heavy use of Julia’s multi-processing capabilities. I am trying to keep dependency bloat to a minimum and I never quite understand why I have to type `instantiate` so often. I don’t know how it’s all going to work(?) when I come to use the Package Compiler system, because, obvs., it can’t suddenly do a JIT comple in the middle of something scary. If you or any one else can recommend a good (non-beginner’s, seriously big project) tutorial I would be so happy. 😁

---

<div class="post-metadata">

**Author:** ![gustaphe](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gustaphe/32/18174_2.png) [@gustaphe](https://discourse.julialang.org/u/gustaphe)\
**Post date:** [October 30, 2021, 1:57pm UTC](https://discourse.julialang.org/t/use-package-not-in-current-environment-for-testing-development/70424/13 "2021-10-30T13:57:30Z")

</div>

Sounds like you want [Tutorial on precompilation](https://julialang.org/blog/2021/01/precompile_tutorial/) (but there will always be a risk of “a JIT comple in the middle of something scary” - I don’t think there’s any guaranteeing against it)