ANN: a solution to the precompilation problem: JuliaManager.jl / jlm CLI, a system image manger for Julia

Package Announcements
,
1

I wrote a command line program jlm to fix some pains in using Julia:

  • We can create multiple projects with new Pkg.jl quite easily. However, switching projects is painful because it causes re-precompilation of the packages. jlm fixes it by using different precompilation cache for different project (actually any directory once you run jlm init) [1].

  • MKL.jl is a very convenient way to use MKL with Julia. However, using it with standard Julia installation is problematic due to the precompilation issue. See the documentation for how to use jlm with MKL.jl.

  • It’s possible to create system images to contain a custom set of packages using PackageCompiler.jl. However, again, switching system images invokes the precompilation issue. You can avoid it if you compile the system images in a jlm-compatible way. I wrote a few recipes to create such system images here: GitHub - tkf/julia-sysimage-recipes: Recipes to build system images for Julia. (But this feature needs some documentation.)

Quick usage quoted from README:

(v1.2) pkg> add JuliaManager
...

julia> using JuliaManager

julia> JuliaManager.install_cli()
...

You need to add ~/.julia/bin to $PATH as would be indicated if it
not. Then, you can use jlm:

$ cd PATH/TO/YOUR/PROJECT

$ jlm init  # first run will be slow
...

$ jlm run
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.1.0 (2019-01-21)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia>

See jlm --help or the documentation for more
information.

jlm still is a quite alpha software. It does not support Windows at the moment (although it shouldn’t be difficult) and only tested in Linux. jlm command is implemented in Python and requires Python ≥ 3.5. JuliaManager.jl is tested with Julia 1.0, 1.1, and 1.2-DEV.

[1] This is an implementation of my early proposal RFC: a "workaround" for the multi-project precompilation cache problem without long-term code debt. It turned out I can do this without waiting for my PR to be merged if I “gorilla-patch” the system image. Furthermore, I improved what I proposed a bit so that you don’t need to create a system image for each project (which may take some time). Note that, even though gorilla sounds stronger than monkey, jlm never modifies the original system image. It is designed to not interrupt standard julia usage; if you invoke julia command as usual there should be no effect of jlm.

17 Likes