Easy packaging of a complete environment for running a course?

General Usage
1

One solution for universities that cannot pony up the use fees for juliabox could be to make it really easy for the instructor to set up the environment just the way the instructor wants it (with the packages that are required for the course, all precompiled and ready to go), and then to make a bundle out of that installation that can be easily transferred to the student’s computers. Perhaps such a thing is already possible?

2 Likes
2

If the computers have internet access and if you can let go of the “precompiled and ready to go” requirement, you could:

  1. create a fresh environment using julia --project=. in a new folder, then add all the packages you’ll need for the class.
  2. distribute the Project.toml and Manifest.toml files that Pkg creates in that new directory to your students and ask them to run julia --project=. in a directory containing the toml files, then pkg> instantiate and pkg> precompile.

By distributing the Manifest.toml in addition to the Project.toml, everybody will be using the same versions of the packages (perhaps you could even pin the packages), and you can also include unpublished packages containing course-specific code by just pkg> adding them with a URL.

If having students precompile the code themselves is not an option, I think making a custom system image is the only option, which is a lot more involved.

11 Likes
3

True, I thought of that. However, I am wondering about the possibility of creating an installation like this and then zipping up all the files that such process generates. Would it be possible to unzip that folder tree and get a functional Julia installation? I will attempt to do this when I find some time, and report here. But perhaps someone has already tried this?

4

Maybe a Docker image could be a solution?

5 Likes
5

Has been a big success for this. You version a package and Manifest, and then in any code or jupyter notebook, you make a single call to activate itit (where it downloads and instnatuates if required)

8 Likes
6

Is there a simple explanation of how this works?

7

@arnavsood can fill in more details, but basically all it does is download a github repo with a project and manifest (if necessary) and activate it. If it hasn’t been downloaded before, it instantiates and precompiles.

So you just put the same line of code at the top of all notebooks. If you want to change a version of the package and Manifest, you just tag a different version in the repo.

8

Can one precompile in the build script, or is that a hack?

9

What build script are you referring to?

Edit: Ah, deps/build.jl?

10

It’s a hack as far as I’m concerned! Precompile works best when called once, after every package got installed. If each package precompiles itself on build, it would likely lead to worse precompilation behaviour.

1 Like
11

Why not just make your own depot, install all the packages you need within there (and instantiate or not depending on whether you want it to be tiny or huge), tar+gz or zip it and send it off to students? I don’t think you’re going to get around doing the initial precompile of all packages, but how much time does that really buy you anyway? 5 minutes maybe?

Edit: Note that making your own depot is as simple as running JULIA_DEPOT_PATH=/my/depot/path julia and running ] up.

2 Likes
12

@jlperla basically summed it up, but a few things worth mentioning on the InstantiateFromURL approach:

  1. We have an option to put these packages into a user’s “default” environment (i.e., ~/.julia/environments/v1.0) so that they can play around on their own without needing the custom code.

  2. We have a force option, so that if students ever mess up a given environment, they can easily do a “factory reset.”

  3. You can use GitHub version information, corresponding to releases (or commits) of the TOML repo in question.

For an example of applications, you can see the lectures at `lectures.quantecon.org/jl/mccall_model.html (for example), with the corresponding GitHub.com/quantecon/quanteconlecturepackages TOML.

Happy to help out/explain in more detail if that would be useful.

2 Likes
13

Cheers,

Given the above posts from 2019, I wonder if the topic is already addressed… can anyone please comment?

My use case is as follows:

  • Main Julia instance is a Linux desktop environment, AArch64, where all heavy workload is executed.
  • Secondary Julia instances are Linux AArch64 IoT devices, where the workload for precompiling might be too much (even if a one-shot compilation).

I managed to mirror the hidden “.julia” folder from main to secondary by using rsync as follows:

rsync -azP --delete <origin> <destination>

After that, Julia has worked out-of-the-box at the target IoT devices, except for the precompilation: at the first call of an “using” code statement, system started to precompile ALL packages.

Any advise on how to alleviate this workload is appreciated. Thanks in advance.