(very) Long compilation time when adding packages to Google Colab

General Usage
1

I really enjoy Julia’s support on Google Colab!
One annoying issue is that adding packages (even simple ones) leads to an endless (pre (?)-)compilation process.
For instance:

import Pkg; Pkg.add(["Bootstrap"]);

still running after 15 min of wait…

Is there a workaround?

Thanks!

1 Like
2

I also encountered this problem several days ago. In my case adding a package from Github ended up taking more than 30 minutes (I gave up after around 30 minutes).

Probably you can try activating new environment before adding the packages.

Pkg.activate("NewEnv", shared=true)
Pkg.add("Bootstrap")

I just tried this and in my case the precompilation time is faster.

3

There are env variables you can set which control what packages are allowed to update when installing new ones. You can pick one of the more conservative ones which should avoid re-precompiling all the packages shipped with the default env. Quoting from Pkg.jl’s changelog:

  • To reduce the amount of time spent downloading and precompiling new package versions when working with multiple
    environments, a new preserve strategy PRESERVE_ALL_INSTALLED has been added which will preserve all existing
    dependencies and only add versions of the new packages that are already installed. i.e. pkg> add --preserve=installed Foo.
    Also a new tiered resolve strategy PRESERVE_TIERED_INSTALLED that tries this first, which can be set to the default
    strategy by setting the env var JULIA_PKG_PRESERVE_TIERED_INSTALLED to true ([#3378]).
1 Like
4

For the noobs among us, please provide explicit instructions or perhaps share the link to a couple preconfigured notebooks (one with CUDA and one without?) with the absolute minimal that we can copy and extend upon.

I am really happy that Google is providing this service. With a little extra attention we can make it very practical and useful in learning and exploration. Not all of us need Lux and Enzyme and Rmath etc, we could save a lot of time in compute and downloads by being able to extend an empty Project per notebook.