Handling multiple versions of Julia

Tooling
1

I’m curious how folks are handling multiple versions of Julia.

On Linux, I’ve been putting the generic binaries in a common folder (~/.julia_versions).

direnv works nicely as a way to customize which version of Julia is loaded on a per-folder basis. That’s also nice as a way to document which version of Julia was used with the code. @garrison has a PR to build in Julia support. You can also add support manually by adding the following to ~/.direnvrc:

layout_julia() {
  export JULIA_PROJECT=$PWD
}

use_julia() {                                                                    
  load_prefix $HOME/.julia_versions/julia-$1
}

A typical .envrc for Julia looks like:

layout julia
use julia 0.7.0

Overall, this is really nice. It particularly works well with the new package manager. On Windows, I’ve gotten this to work with Git bash, but you need a workaround.

For more general launching of different versions, I’ve got the following in my .bashrc:

julia_version() {
    ~/.julia_versions/julia-$1/bin/julia
}

alias j=julia
alias jv=julia_version
alias j6="jv 0.6.4"
alias j7="jv 0.7.0"
alias j1="jv 1.0.0"

That works, but it’s a little clunky.

2 Likes
2

I use modules to switch between julia binaries and a self-compiled MKL version.

Example lua file:

help([==[

Description
===========
Provides access to Julia 0.6.4 binaries

]==])

whatis([==[Description: Provides access to Julia 0.6.4 binaries]==])

unload("julia-mkl")

load("Intel/2018.2.199-GCC-5.5.0", "ParaStationMPI/5.2.1-1", "HDF5/1.10.1")

setenv("SSL_CERT_FILE", "/etc/pki/tls/cert.pem")
setenv("EDITOR", "vim")
setenv("JULIA_PKGDIR", "/gpfs/homea/hku27/hku273/.julia")

prepend_path("LD_LIBRARY_PATH", "/gpfs/homea/hku27/hku273/software/julia/0.6.4/usr/lib")
prepend_path("CPATH", "/gpfs/homea/hku27/hku273/software/julia/0.6.4/usr/include")
prepend_path("PATH", "/gpfs/homea/hku27/hku273/software/julia/0.6.4/bin")

After adding the folder with those files to my $MODULEPATH I can just do module load julia/0.6.4 or module load julia-mkl/0.6.4.

2 Likes
3

Thanks. Hadn’t seen modules before.

4

In the past I’ve always had a bunch of symlinks in ~/bin which point to the different versions of julia compiled on my system. With this, putting e.g. #!/usr/bin/env julia-0.6 at the top of a file would both remind me what version of julia the code is written for and also just work (at least machines I use, where the symlinks exist). Following your post to that pull request, I’ve been exploring using direnv instead for this purpose, and it’s been working quite well so far.

I’ve always thought that it might be useful for the julia binary to include its version number, just as I do with my symlinks locally. I’m reminded of all the work and debates behind python’s PEP 394 – seems as if it would have gone a lot smoother had there been such guidelines prior to the release of python3 ;-).

5

Technically off topic to this thread, but relevant to general workflow considerations, particularly using direnv: Here is how my .envrc files are converging:

export JULIA_PROJECT=$PWD
export JULIA_LOAD_PATH=:$PWD/src
PATH_add bin

The first line sets JULIA_PROJECT, just as my pull request to direnv does. The second line allows me to create modules in files src/Module1.jl, src/Module2.jl, etc. and import them easily. I’m still not sure what the convention should be for this. (Should this directory instead be called modules perhaps? I think I prefer src to match the directory layout of packages, but am open to comments.) And finally, the last line allows me to put my actual executables with the shebang line in the bin directory and have them always be in the PATH.

6

Very similarly to your approach, I just have symlinks julia1, julia07, and julia06, also a juliadev for whatever I have compiled in the git repo.

I just start whatever I need, these days I use mostly 0.7 (for upgrading existing packages) or 1.0 (for production work).

1 Like
7

I’m a lot less sophisticated. I have directories

~/julia0.5
~/julia0.6
~/julia0.7
~/julia1.0
~/julia

which all contain clones of Julia’s git repo, where all but the last are parked on a release tag and the last follows master periodically. None of them is in PATH and I just start whichever I want at the time.

1 Like
8

The dream here seems like it might be to use BinaryBuilder for Julia itself, record Julia versions in the manifest file and have julia be a thin wrapper that loads the correct version of Julia so that you can control that just as you can versions of packages. In the meantime, any ideas for features that could make things easier in terms of running different julias are definitely welcomed!

5 Likes
9

It would be ideal if such a system were also aware of and able to work with system-installed version(s) of julia (e.g those from a distribution’s package manager). Even if users are encouraged to use the BinaryBuilder versions of julia, at least having things not be in conflict would be very valuable. A lot of users will just want to use whatever the system has already installed, and having a common workflow for both cases would be most welcome (e.g. will make it easier to write good tutorials/documentation that work universally).

10

Yes, I think you’d want a configurable map from Julia version numbers and/or version hashes to executable locations. The BinaryBuilder approach is mostly good for getting a version you don’t already have without hassle.

1 Like
11

Hi,

One tip is to use export JULIA_PROJECT=@. The @. is a special value interpreted by Julia to mean the current working directory. So then you can set this at a more global place instead of setting it in each project directory.

Glen

3 Likes
12

I define aliases in a ~/.bashrc or ~/.bash_aliases file:

alias jdev="/home/chris/Documents/prog/julia-dev2/usr/bin/julia -O3"
1 Like
13

In case anyone else is browsing this topic looking for options:

The jlenv org contains a collection projects addressing the various use cases that give rise to needing to switch between different versions of Julia. These cover a range of use cases:

  • chjulia: just switch julia versions: Simple switch and run in prod and dev.
  • jlenv & plugins: More elaborate setups, but not complex or slow moving enough to justify the Chef/Ansible/Salt overhead.
  • jlenv-cookbook: Manage more complex slow moving setups where there are some audit requirements such as ChefSpec/InSpec/ServerSpec, or their Salt/Ansible equivalents.

An announcement in the Tools forum has more details.