4. Working with Environment · Pkg.jl (julialang.org):
A “shared” environment is simply an environment that exists in
~/.julia/environments. The defaultv1.8environment is therefore a shared environment.
Shared environments have a@before their name in the Pkg REPL prompt.
Is there anyone who can tell me what’s the essential difference between a “shared” environment and a general one, and when should I use a “shared” environment?
And, what use is the original @v1.8 environment suggested for?
I think the most salient difference is that you can easily activate a shared environment without memorizing the path to a particular directory. Normal environments are activated by referring to their filesystem path, i.e., julia --project=/path/to/environment (though usually you’ll cd into the directory and run julia --project=.). Shared environments can be activated as julia --project=@name regardless of what your current directory is.
I currently have one shared environment in addition to @v1.x. I call it @scratch, and it contains packages that come in handy for quick, disposable analyses and plots in my day-to-day. That way, I can run julia --project=@scratch from any folder and be ready to go if I want to make a quick plot or benchmark et cetera. I’ve also installed an IJulia kernel that launches julia with @scratch activated, so I can just as easily open a scratch notebook anywhere.
The packages installed in @scratch reflect what sorts of one-off things I’ve been doing over the last several months. Currently, the list is:
(@scratch) pkg> status
Status `~/.julia/environments/scratch/Project.toml`
[ec485272] ArnoldiMethod v0.2.0
[7d9fca2a] Arpack v0.5.4
[69666777] Arrow v2.6.2
[336ed68f] CSV v0.10.11
[052768ef] CUDA v4.3.2
[13f3f980] CairoMakie v0.10.6
[082447d4] ChainRules v1.51.0
[a93c6f00] DataFrames v1.5.0
[7034ab61] FastBroadcast v0.2.5
[5789e2e9] FileIO v1.16.1
[f6369f11] ForwardDiff v0.10.35
[e9467ef8] GLMakie v0.8.6
[14197337] GenericLinearAlgebra v0.3.11
[f67ccb44] HDF5 v0.16.15
[34004b35] HypergeometricFunctions v0.3.17
[42fd0dbc] IterativeSolvers v0.9.2
[4138dd39] JLD v0.13.3
[033835bb] JLD2 v0.4.31
[b9914132] JSONTables v1.0.3
[11c68b98] JacobiDavidson v0.1.1
[0b1a1467] KrylovKit v0.6.0
[5078a376] LazyArrays v1.1.1
[7a12625a] LinearMaps v3.10.1
[bdcacae8] LoopVectorization v0.12.162
[0987c9cc] MonteCarloMeasurements v1.1.4
[1dea7af3] OrdinaryDiffEq v6.53.2
[91a5bcdd] Plots v1.38.16
[f517fe37] Polyester v0.7.3
[92933f4c] ProgressMeter v1.7.2
[1fd47b50] QuadGK v2.8.2
[276daf66] SpecialFunctions v2.3.0
[90137ffa] StaticArrays v1.5.26
[789caeaf] StochasticDiffEq v6.61.1
[09ab397b] StructArrays v0.6.15
[856f2bd8] StructTypes v1.10.0
[b36ab563] TaylorDiff v0.2.1
[b189fb0b] ThreadPools v2.1.1
[bc48ee85] Tullio v0.3.5
[b8865327] UnicodePlots v3.6.0
[e88e6eb3] Zygote v0.6.62
[2f01184e] SparseArrays
[4607b0f0] SuiteSparse
The base environment @v1.x is always present in addition to whatever other environment you’ve activated, unless you jump through extra hoops to avoid this. Therefore, it’s recommended to keep @v1.x minimal and only use it for packages that you want to have available anywhere, and that don’t really belong in any given package or project environment. For the most part, that means developer tools—things like Revise, BenchmarkTools, ProfileView, IJulia. My current list is:
(@v1.9) pkg> status
Status `~/.julia/environments/v1.9/Project.toml`
[6e4b80f9] BenchmarkTools v1.3.2
[f68482b8] Cthulhu v2.9.1
[7073ff75] IJulia v1.24.2
[33e6dc65] MKL v0.6.0
[9b87118b] PackageCompiler v2.1.7
[14b8a8f1] PkgTemplates v0.7.38
[c46f51b8] ProfileView v1.7.1
[e0db7c4e] ReTest v0.3.2
[295af30f] Revise v3.5.3
[1e6cf692] TestEnv v1.100.0
Note that by installing MKL in the base environment and putting using MKL in my ~/.julia/config/startup.jl, I’m able to select MKL as the BLAS/LAPACK backend globally on this machine. This is another example of something that I always want on this computer, but that is not the business of any particular package or project (imagine I share a project with a friend who’s on Apple or AMD hardware—I definitely don’t want the project itself to have an opinion about which backend to use).
Thanks! Your answer is very helpful to me!
Now I still have a question: The packages installed in the base environment @1.x or other shared environments such as your @scratch can also be used in any other customized environments, can they?
For example, if you set up a new environment called NewEnv:
(@v1.8) pkg> activate NewEnv
[ Info: activating new environment at `~/NewEnv/Project.toml`.
(NewEnv) pkg>
and using the package UnicodePlots that was installed in your @scratch environment:
julia> using UnicodePlots
will this be legal?
No, the default load path is such that the first place to look for a package is the currently activated environment, then the @v1.X environment of your Julia version, then the standard libraries of this Julia installation. The environment @shared would not be found, although you can in theory mutate your load path variable and add it to the stack. That’s usually not a good way to do things though. Best is to keep the local environments clean and specific to the task you need them for. And leave only essential tools you always need but which don’t carry tons of dependencies themselves in the @v1.X env.
No, shared environments are not included in your load path unless you activate them explicitly. In this sense, they behave the same as regular environments. The only special thing about shared environments is that you can activate them by name instead of path—in every other sense, they work like your regular environments.
The exception is the base environment @v1.x, which is always available unless you take specific steps to remove it from the load path. This is where you can put packages you want to always have available. See Code Loading · The Julia Language.
You can in principle muck around with the load path to achieve arbitrary environment stacks (see the documentation for LOAD_PATH and Base.load_path), but environment stacking is brittle and rarely a good way to organize your workflow. In particular, the loading of dependencies does not respect environment boundaries, so you may end up loading incompatible versions of packages. Imagine you stack NewEnv on top of @scratch and @scratch contains package A which depends on package B version 1, while NewEnv contains package B version 2. Now you can do using A to load A from @scratch, but when the code in A does using B, the version of B that gets loaded is the one in NewEnv, which is incompatible with A. This is one of the reasons it’s a good idea to keep the base environment minimal and only use it for the packages that help you develop and work with your code, rather than the packages your code actually relies on to do its job (for example, create plots).
In short, if you want to use UnicodePlots within NewEnv, then it belongs in NewEnv.
Very clear. Thanks!
How does any of this change if I am using VSCode instead of the REPL? In particular I often start Julia by going ‘Shift+Enter’ on a line of code. What environment does this activate?
The default should be the base environment @1.x. If you want to activate a new environment in VS Code’s editor, maybe you should type:
using Pkg
Pkg.activate("YourNewEnv")
In VSCode, in the bottom right, you can click to choose the environment.
May I ask, how to deactivate an activated user-defined environment, and how to delete one?
To “deactivate” you actually activate another environment, for instance the main global one if no specification is given:
(@v1.9) pkg> activate --temp
Activating new project at `/tmp/jl_tSQluH`
(jl_tSQluH) pkg> activate
Activating project at `~/.julia/environments/v1.9`
(@v1.9) pkg>
To delete the environment, delete the corresponding directory (shown there) manually, in the case here /tmp/jl_tSQluH.
I see. Thanks!