# Speculations about the default environment (or a new draft environment)
**URL:** https://discourse.julialang.org/t/speculations-about-the-default-environment-or-a-new-draft-environment/78967
**Category:** Internals & Design
**Tags:** environment
**Created:** [April 3, 2022, 11:34pm UTC](https://discourse.julialang.org/t/speculations-about-the-default-environment-or-a-new-draft-environment/78967 "2022-04-03T23:34:51Z")
**Posts on this page:** 20
**Page:** 3
### Author:  [@stillyslalom](https://discourse.julialang.org/u/stillyslalom)
#### Post date: [April 6, 2022, 6:06pm UTC](https://discourse.julialang.org/t/speculations-about-the-default-environment-or-a-new-draft-environment/78967/42 "2022-04-06T18:06:50Z")
I’ll illustrate a little vignette of how global environment bloat becomes painful for new users:
- Make a few simple plots based on a tutorial using a handful of packages installed in the global environment
- As the project’s scope expands, gradually add packages
- Perform a throwaway analysis for a separate project that requires one or two more packages (also added to the global environment)
- A third project appears. This time, two of the necessary packages are incompatible, but a fix is available on `#master`
- This breaks something deep inside the graphics stack for the first project
- [two hours of package resolution hell and refactoring just because the user wanted to make a plot]
All this pain would be avoided, of course, if users created a new environment for each project, but as multiple commenters have mentioned, the necessity of siloing projects is not drilled into new users who are happy to take the path of least resistance. The problem isn’t the existence of a persistent global environment, but rather that the global environment is the default. It shields new users from the complexity of Pkg, projects, manifests, environments, etc., but that debt can only be deferred for so long before it needs to be collected (with interest).
---
### Author:  [@mbauman](https://discourse.julialang.org/u/mbauman)
#### Post date: [April 6, 2022, 6:10pm UTC](https://discourse.julialang.org/t/speculations-about-the-default-environment-or-a-new-draft-environment/78967/43 "2022-04-06T18:10:56Z")
I do think some simple UX things like [Pkg.jl#961](https://github.com/JuliaLang/Pkg.jl/issues/961) would significantly improve things here without any fundamental changes.
[https://github.com/JuliaLang/Pkg.jl/issues/961](https://github.com/JuliaLang/Pkg.jl/issues/961)
Speaking personally, I regularly do quick and ad-hoc testing in my default global environment, adding packages willy-nilly. It’s quick, convenient and easy. But then I’ll also remove them, either when I’m done or at some other point down the road. And I’ll take the time to throw things into a project environments the moment it goes from REPL play into a `.jl` file. If I’m touching the filesystem then I have a directory for it and it means I might come back to it.
---
### Author:  [@tbeason](https://discourse.julialang.org/u/tbeason)
#### Post date: [April 6, 2022, 6:13pm UTC](https://discourse.julialang.org/t/speculations-about-the-default-environment-or-a-new-draft-environment/78967/44 "2022-04-06T18:13:21Z")
I have begun to do most of my simple checks (either for MWE for the forum or for my own play) only after `] activate --temp`. But this not really a great solution in my opinion.
---
### Author:  [@suavesito](https://discourse.julialang.org/u/suavesito)
#### Post date: [April 6, 2022, 7:03pm UTC](https://discourse.julialang.org/t/speculations-about-the-default-environment-or-a-new-draft-environment/78967/45 "2022-04-06T19:03:09Z")
> [@lmiq](#):
>
> By default, installing anything there may trigger a cascade of updates of everything. Also very inconvenient if the purpose is to do some quick exploratory code.
A solution might be to have a flag like `--constrained`. Using it in package installation would force to look for the latest version of `PackageToBeInstalled` that does not implies an update on any other package of the current environment.
This would not change the separation of generating the `Manifest.toml` and loading the package, and the version resolution would occur in package installation. I guess this still have the following problem.
> [@StefanKarpinski](#):
>
> That’s very likely to frequently back users into corners where they want to load **(install)** some package but there is no version that’s possible.
(Bold is mine.)
It is also a really hard problem to solve and implies a _lot_ of work. I guess similarly as @StefanKarpinski says here (I know this is for loading, but the work I guess is similar for installing).
> [@StefanKarpinski](#):
>
> If we want to not load known incompatible versions of packages, it’s worse: then we need to load all registries and parse the Compat TOML files for `Package` _and_ for every package that we’ve loaded so far to make sure that they are compatible.
For the following I think that making a GUI-like tool would help a lot to solve the problem, it is a graph dependency visualization utility, so should not be that hard to make. 🙂
> [@lmiq](#):
>
> From time to time, one of these packages fail to keep the Compat entry up to date, and then it holds back the updating of other packages, finally leading to a compatibility roadblock. Finding which package is holding other back is not that straightforward.
---
### Author:  [@lmiq](https://discourse.julialang.org/u/lmiq)
#### Post date: [April 6, 2022, 7:08pm UTC](https://discourse.julialang.org/t/speculations-about-the-default-environment-or-a-new-draft-environment/78967/46 "2022-04-06T19:08:45Z")
I just noticed that when one installs a package that depends on another package, but that other package is loaded in a incompatible version from the shared environment, we get this very gentle warning:
```julia
1 dependency precompiled but a different version is currently loaded.
Restart julia to access the new version.
```
So it seems that _that_ problem is already present in the current workflow. Does not seem different to me, and a warning / error message may suffice.
---
### Author:  [@suavesito](https://discourse.julialang.org/u/suavesito)
#### Post date: [April 6, 2022, 7:13pm UTC](https://discourse.julialang.org/t/speculations-about-the-default-environment-or-a-new-draft-environment/78967/47 "2022-04-06T19:13:50Z")
I normally use temporary environments for all my try-and-trash code, I even have a little script to open Julia in that mode (with `OhMyREPL` and `TerminalPager`).
This is it
```julia
julia -q --startup-file=no -e 'using OhMyREPL, TerminalPager; ENV["PAGER"] = "julia_pager.sh"; using Pkg; Pkg.activate(temp=true)' -i
```
where `julia_pager.sh` is
```julia
#!/usr/bin/env bash
bat -l julia -n --color=always "$2" | less -R $1
```
---
### Author:  [@davidanthoff](https://discourse.julialang.org/u/davidanthoff)
#### Post date: [April 6, 2022, 7:15pm UTC](https://discourse.julialang.org/t/speculations-about-the-default-environment-or-a-new-draft-environment/78967/48 "2022-04-06T19:15:37Z")
> [@StefanKarpinski](#):
>
> You’re really relying on the assumption that only people who know what they’re doing would put anything in `@v#.#-global` at all and that they would presumably avoid depending on things in there from their scripts.
Yes, that is exactly right. I’m not claiming it would solve all problems, but I think it is a fairly simple (and hopefully not too controversial or disruptive) change that would really, really help a ton. I think it would essentially just align the actual behavior with what most users think the implementation is (or at least the vast majority of new to medium experienced users). So my vote would be that we try that (or something similar) for say 1.9, and then we can still have a long and extensive discussion for a perfect solution, but in the meantime we have something good 🙂
---
### Author:  [@lmiq](https://discourse.julialang.org/u/lmiq)
#### Post date: [April 6, 2022, 7:23pm UTC](https://discourse.julialang.org/t/speculations-about-the-default-environment-or-a-new-draft-environment/78967/49 "2022-04-06T19:23:05Z")
That, plus starting julia in a temporary environment more quickly, such has
```julia
julia --temp
```
plus a macro to install offline packages only (`@reuse` or so) is pretty much what the `Draft` environment looks like.
(saving the environment + history would be a must)
---
### Author:  [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)
#### Post date: [April 6, 2022, 7:59pm UTC](https://discourse.julialang.org/t/speculations-about-the-default-environment-or-a-new-draft-environment/78967/50 "2022-04-06T19:59:35Z")
> [@lmiq](#):
>
> How is this different from what may happen when loading a package available in the `@v1.7` shared environment when using another environment? Shouldn’t the possible issues be exactly the same? Sincere question, here, is there any difference? My current impression is that the package available on the shared environment is just loaded for good.
It is true that when a package is auto-installed into an environment, it cannot change the versions of packages that are already loaded. However, the issue is limited to only _new_ packages you try to load into the environment. That is relatively uncommon since the packages you commonly use are already in your default environment with versions that have already been resolved to be compatible.
> [@lmiq](#):
>
> I just noticed that when one installs a package that depends on another package, but that other package is loaded in a incompatible version from the shared environment, we get this very gentle warning:
>
> ```julia
> 1 dependency precompiled but a different version is currently loaded.
> Restart julia to access the new version.
>
> ```
>
> So it seems that _that_ problem is already present in the current workflow. Does not seem different to me, and a warning / error message may suffice.
The difference is that when you restart Julia it starts with a correctly resolved set of package versions _including_ the one you just added and got this warning for, so when you do the same sequence of imports you don’t get an error and have compatible versions of all the packages. In your proposal on the other hand, the environment is ephemeral, so when you restart Julia and do the same sequence of imports, you get into the exact same broken state with the same warning because there is no memory of wanting that set of packages. Permanence of the environment is exactly the feature that allows for not repeating the same mistake over and over again.
Note that there can be situations where even though there is a compatible set of versions of the packages you want to use there is _no possible order_ of package imports that leads to a compatible set of versions if you commit to the best version of each package before considering the other packages. You need to have a global view of the set of packages that you want to use when picking versions in order to be able to find all possible solutions. Here’s an example:
- you want to use packages A and B
- A has versions [2, 1]
- B has versions [2, 1]
- C has versions [3, 2, 1]
- A and B both depend on C
- A (all versions) is incompatible with C3
- B (all versions) is incompatible with C2
Here’s the conundrum:
- If you load A first, you get A2 and C2 which is incompatible with B;
- If you load B first, you get B2 and C3 which is incompatible with A.
If you can reason about it globally, however, you can see that A1, B1, C1 is a compatible set of versions, it’s just not one you can find with a greedy algorithm.
---
### Author:  [@lmiq](https://discourse.julialang.org/u/lmiq)
#### Post date: [April 6, 2022, 8:11pm UTC](https://discourse.julialang.org/t/speculations-about-the-default-environment-or-a-new-draft-environment/78967/51 "2022-04-06T20:11:39Z")
I appreciate the time you take to explain those things, and I really feel uncomfortable following the discussion (given that surely you have better things to do). But the changes in “my proposal” to adapt to that could be very simple (and maybe even could be the default behavior): just throw an error if an incompatibility is found, suggesting perhaps the user to update things.
Actually this could be the behavior now, and not letting the user in a possibly broken Julia session (I’m not claiming it _should_ be, but it seems that could well have been an acceptable decision with other tradeoffs).
(can we have the`julia --temp` flag? 😊 )
---
### Author:  [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)
#### Post date: [April 6, 2022, 8:16pm UTC](https://discourse.julialang.org/t/speculations-about-the-default-environment-or-a-new-draft-environment/78967/52 "2022-04-06T20:16:34Z")
We do have that option; it’s spelled `julia --project=$(mktemp -d)`. I suppose we could have a shorter spelling for it.
---
### Author:  [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)
#### Post date: [April 6, 2022, 8:23pm UTC](https://discourse.julialang.org/t/speculations-about-the-default-environment-or-a-new-draft-environment/78967/53 "2022-04-06T20:23:36Z")
The biggest issue with doing this is that there is no syntax in `JULIA_LOAD_PATH` for expressing that you want `@v#.#` in the load path only if the active project entry `@` doesn’t resolve to anything. The default value is `@:@v#.#:@stdlib`. What you’re suggesting would be making the default something like this instead: `@|@v#.#:@v#.#-global:@stdlib`, with the assumption that we make `|` special syntax for “the first of these that exists”.
---
### Author:  [@cjdoris](https://discourse.julialang.org/u/cjdoris)
#### Post date: [April 6, 2022, 8:34pm UTC](https://discourse.julialang.org/t/speculations-about-the-default-environment-or-a-new-draft-environment/78967/54 "2022-04-06T20:34:50Z")
It’s not quite the same thing, doesn’t Julia delete temporary projects at exit?
---
### Author:  [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)
#### Post date: [April 6, 2022, 8:35pm UTC](https://discourse.julialang.org/t/speculations-about-the-default-environment-or-a-new-draft-environment/78967/55 "2022-04-06T20:35:37Z")
That’s a feature—you can restart Julia in the same environment if you need to 😁
---
### Author:  [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)
#### Post date: [April 6, 2022, 8:44pm UTC](https://discourse.julialang.org/t/speculations-about-the-default-environment-or-a-new-draft-environment/78967/56 "2022-04-06T20:44:46Z")
Actually, another way to approach this is to change the default active project from nothing to `v#.#` which seems simpler.
---
### Author:  [@davidanthoff](https://discourse.julialang.org/u/davidanthoff)
#### Post date: [April 6, 2022, 9:01pm UTC](https://discourse.julialang.org/t/speculations-about-the-default-environment-or-a-new-draft-environment/78967/57 "2022-04-06T21:01:50Z")
> [@StefanKarpinski](#):
>
> Actually, another way to approach this is to change the default active project from nothing to `v#.#` which seems simpler.
Yes, that was my idea: always activate v1.7 (unless of course a project is specified via the `JULIA_PROJECT` env var or command line flags), and just do entirely away with the situation where there is no active project. And then the `JULIA_LOAD_PATH` would be `@:@v#.#-global:@stdlib`.
That might even simplify things in the code base more generally because one could now rely on there always being an active project, i.e. no longer would one need to special case handle the situation where there is no active project.
---
### Author:  [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)
#### Post date: [April 6, 2022, 9:30pm UTC](https://discourse.julialang.org/t/speculations-about-the-default-environment-or-a-new-draft-environment/78967/58 "2022-04-06T21:30:57Z")
First of all, I’m glad to see that there is a chance to have this changed this time!
I’m wondering what could be a better name for what is currently called `v#.#-global` here. IMO, the global is misleading because the regular `v#.#` environment is also global in some sense (irrespective of where one starts Julia, it get’s automatically loaded if no explicit active project is specified),
> [@StefanKarpinski](#):
>
> If you really want to enforce that scripts in active projects are standalone, then you can’t allow even that.
Would it make sense to consider introducing a command line option that simply disables the stacking in the sense that it drops `@v#.#-global` from the default `LOAD_PATH` stack when asked for?
> [@StefanKarpinski](#):
>
> We do have that option; it’s spelled `julia --project=$(mktemp -d)` . I suppose we could have a shorter spelling for it.
Is `mktemp` available on Windows?
---
### Author:  [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)
#### Post date: [April 6, 2022, 9:58pm UTC](https://discourse.julialang.org/t/speculations-about-the-default-environment-or-a-new-draft-environment/78967/59 "2022-04-06T21:58:18Z")
Yeah I didn’t want to bike shed the name here so I just went with David’s suggestion to explore the concept. Stacking can already be disabled by setting `JULIA_LOAD_PATH=@`. Who knows if Windows has mktemp? Not me.
---
### Author:  [@davidanthoff](https://discourse.julialang.org/u/davidanthoff)
#### Post date: [April 6, 2022, 11:46pm UTC](https://discourse.julialang.org/t/speculations-about-the-default-environment-or-a-new-draft-environment/78967/60 "2022-04-06T23:46:07Z")
> [@StefanKarpinski](#):
>
> Yeah I didn’t want to bike shed the name here
Yep, and I’m not really sure whether “global” is a good name for that either, I just picked something 🙂
I think if we were starting from scratch, it might make sense to call all projects in `.julia/environments` “global” and the one that is always loaded “shared”, but of course that is no longer an option, because the term “shared” is already used for everything in the `.julia/environments` folder, so ideally we would find a good new name for the one that would always be loaded 🙂
---
### Author:  [@jmair](https://discourse.julialang.org/u/jmair)
#### Post date: [April 7, 2022, 6:55am UTC](https://discourse.julialang.org/t/speculations-about-the-default-environment-or-a-new-draft-environment/78967/61 "2022-04-07T06:55:21Z")
> [@lmiq](#):
>
> Well, my experience is that:
>
> 1. After a while, if there are too many packages there, installing or updating anything starts to become very slow. Some packages are simply impossible to keep available in a shared environment (CUDA, in particular), because having there implies that I may get an update of that at some time and have to wait several minutes to work on what I wanted.
> 2. By default, installing anything there may trigger a cascade of updates of everything. Also very inconvenient if the purpose is to do some quick exploratory code.
I experience this almost weekly. What’s unfortunate is that most of the time I am loading the same packages (Plots, CUDA etc), and the latency involved feels unnecessary.
Thanks for the proposed workflow, I hope that it speeds up this process.
[Previous page](https://discourse.julialang.org/t/speculations-about-the-default-environment-or-a-new-draft-environment/78967.md?page=2)
[Next page](https://discourse.julialang.org/t/speculations-about-the-default-environment-or-a-new-draft-environment/78967.md?page=4)