I would like to start a thread about best practice for how to set up a centralized Julia installation in a cluster environment or local institution network. Hopefully, this will lead to a tutorial like write-up in the end. I want to focus on Julia 1.0 and, thus, Pkg(3). How does one set up a system…

@htwai Just tested with a clean user environment. set JULIA_PROJECT = "~/.julia/environment/v1.1 ./julia/compiled is actually created. I might not have tested with the JULIA_PROJECT env var in the previous test. Everything looks pretty good for the admin/user pattern if you put your central dep…

Here, with JULIA_PROJECT set, how does a user know which packages are available globally? Running Pkg.installed() returns nothing. Is there a way to specific to look into all depot paths?

You could temporarily switch your project to the central depot. Pkg.activate(DEPOT_PATH[2]*"/environments/v1.1") This switches, finds installed packages and then switches back to your original project: Pkg.activate(DEPOT_PATH[2]*"/environments/v1.1"); installed_pkgs = Pkg.installed(); Pkg.activate…

See also https://github.com/JuliaLang/Pkg.jl/issues/961

Here is what I contributed to the julia-in-production BoF channel at JuliaCon 2020: Hi all, I work as a computational scientist at the Swiss National Supercomputing Centre (CSCS) and am the responsible for Julia computing at CSCS. We have put Julia in September last year in production on the Piz …

[image] samo: Then, this works all pretty well for small or medium jobs, but for large jobs with thousands of GPUS it is not good that each process needs to read from the same files (stacked environment / home). For large-scale deployment, it would be nice if one could create a binary / folder t…

First of all, we are still far from a perfect solution. Besides, like you can see looking at the never-ending discussions in Julia discourse, it is not at all obvious how an ideal solution should look like. We have opted for installing the most important packages for HPC as MPI.jl and CUDA.jl into …

Thanks Sam, that was a good overview. What happens when a new release of e.g. MPI.jl is tagged: do you need to manually go through and add that to the shared depot? Otherwise wouldn’t it install it in the user’s depot?

Sorry for the late reply. I somehow missed it. We install MPI.jl and CUDA.jl together with the Julia base module. Of course, the user can install a newer version of MPI.jl and CUDA.jl in his user space if he needs it; it will take precedence in the stacked environment. Our usual scientific softwar…

How does one set up a centralized Julia installation?

General Usage

samo September 18, 2024, 3:08pm 47

Topic		Replies	Views
Base packages for all users Package Management question	26	2211	March 8, 2022
How to create a system administrator depot? General Usage package-manager	8	2382	September 28, 2018
Installation on managed cluster General Usage cluster , installation	25	1546	July 4, 2023
DEPOT_PATH undefined despite JULIA_DEPOT_PATH being defined in the Environment New to Julia question	27	2009	January 22, 2020
Best practice for packages on shared drives General Usage	12	1238	June 28, 2022

How does one set up a centralized Julia installation?

Related topics