Julia in Colab

[Mason]

That still wouldn’t fix the bigger problem Valentin raised which is

@vchuravy

I noticed that precompilation times are quite bad, the machines seem to be beefy enough so I wonder if the slow down is some unfortunate interaction with the file system?

Ideally one would be able to have a transparent cache for JULIA_DEPOT_PATH. (~/.julia) (That’s something we do on GitHub actions), and support for Project.toml?

[jules]

I assumed some sort of depot did exist, but I didn’t check myself

[MilesCranmer]

@vchuravy

We can disagree on 1 and be of different opinion and that is fine, but the crux for me is 2. because that leads to the question:

Sounds right to me. Indeed it would be fantastic if Colab packaged a precompilation cache, regardless of what packages sit in the base environment.

However one worry I have here is cache invalidation from new packages—note that Colab does not update the packages that frequently IIRC. I think artifacts would be tricky also, in case the new packages require new artifacts. I think a larger base env (with frozen versions), while it isn’t perfect, does mean you can get a start working quicker. Similar to Python—it can take a while to install PyTorch, so I don’t mind that Colab freezes the version at a slightly older value.

[NonDairyNeutrino]

Oops! My brain said “I’m using the +(::CuArray, ::CuArray); I’m good to go.”, but timing just the kernel launch strikes again! I thought it was weird that the GPU line was flat lol; I chocked it up to the GPU might not be saturating even at those longer arrays (which it probably definitely should at 10^8 elements). Thanks for catching that!

I have edited my original post to disclose this and linked to your post!

[asinghvi17]

I believe you can have Javascript run by display(MIME("application/javascript"), yourjs_string), so that can be a “listener” to a message…I was experimenting with this a bit for WGLMakie support but probably wasn’t using the IJulia interface correctly.

[blegat]

There is an issue opened : Support Pluto notebooks for Julia · Issue #5167 · googlecolab/colabtools · GitHub

[metrizable]

Any plans to support the latest stable version of Julia ? (Currently 1.11.3).

On the roadmap, but a little ways out.

We did just upgrade to 1.10.9 following its release a couple days ago. We also pre-install CUDA.jl now (configured to use system-installed CUDA), to save some time.

[metrizable]

@vchuravy

Ideally one would be able to have a transparent cache for JULIA_DEPOT_PATH. (~/.julia) (That’s something we do on GitHub actions), and support for Project.toml?

This is an interesting idea. It’s slightly different than our current experience as @MilesCranmer mentioned, so not sure where it will land, but one we’re thinking on.

[metrizable]

@vchuravy

I noticed that precompilation times are quite bad, the machines seem to be beefy enough so I wonder if the slow down is some unfortunate interaction with the file system?

This seems a bit concerning to me and probably worth filing an issue with a minimal repro.