Production Package Installation Process - Install with user 'a', Access from user 'b'

New to Julia
,
1

Here is the steps on production environment (Very standard)

  1. Deployment process [one time task]: Copy source code and install all packages → this is done by sudo user, lets say “sshuser”
  2. Run time our packages are executed by another service account user → let say “service_user”

In my case user #2 never find the packages installed by user #1

How can I have common environment for both users?

Thanks everyone!

2

before installing packages in step 1, do a popfirst!(DEPOT_PATH)

1 Like
3

If you want to freeze all packages (as per your previous thread) and don’t need users changing anything in the environment, you might consider using PackageCompiler to simply compile all packages into the sysimage - which would also give you quicker startup times!

2 Likes
4

To expand on what @bjarthur said, the location where packages are installed are defined using a variable called DEPOT_PATH. It can be a list of paths, where packages are written to the first element of the list, and read from all the paths in the list.

So on my machine, the default depot path is as follows:

julia> DEPOT_PATH
3-element Array{String,1}:
 "/home/user/.julia"
 "/opt/dev/julia-1.1.0/local/share/julia"
 "/opt/dev/julia-1.1.0/share/julia"

So if you popfirst!(DEPOT_PATH), then your packages will get installed in a location inside your julia install directory, rather than in the user’s home dir.

Now, given this, you can of course set your DEPOT_PATH to any other location on your disk. When you set it from the shell, the environment variable is called JULIA_DEPOT_PATH. To see an example of an install script that does that, look here

Note that generally you would want to make the DEPOT_PATH writeable by any user that runs Julia, since the results of package precompilation (and artifacts) are written to the DEPOT_PATH. For similar reasons, you should not run multiple julia processes simultaneously looking at the same DEPOT_PATH while packages are precompiling. The solution to both is to ensure your packages are precompiled and loaded at install time (with Pkg.precompile() and/or using). Or just run your testsuite at install time.

2 Likes
5

To set your DEPOT_PATH and other environment variables, consider using Modules.

https://lmod.readthedocs.io/en/latest/

6

Also worth looking at easybuild EasyBuild: building software with ease
ans Spack
I found an Easyblock which says it installs Julia packages, but seems a bit out of date
CSCS/julia.py at master · easybuilders/CSCS · GitHub

1 Like
7

Thanks @bjarthur for the quick solution & @avik for the explanation. I tried on my machine and got the concept. Problem is everything works on my machine :joy:

I will try both styles(popfirst and setting JULIA_DEPOT_PATH) on Servers (Linux environment) and update this ticket, but will take some time.

@nilshg - The sysimage is some kind of Julia package image? Complete OS image is not a solution for me.

@johnh - thanks, will look into

8

Yes it’s basically a version of Julia with the packages you’re using built-in, similar to the standard library - so the same way you can just do using LinearAlgebra in a fresh Julia install without having to install package, you could do that for every package in your sysimage.

1 Like
9

@nilshg Wow! Thats nice. Can you direct me to some example/docs?

10

I think the docs are pretty good (NB: I haven’t tried it myself though!): Home · PackageCompiler

1 Like