First off, let me disclaim that this idea partially involves one of the 2 Hard Things in CS [1] (actually maybe the other of them depending how you want to interpret it) and so is probably cursed from the get-go. But it keeps popping into my head every now and then so I’m curious what others might think. And also that I’m a Julia novice and am not acquainted at all to its internals so I’m probably making a lot of incorrect assumptions.
Anyway I was thinking how presumably the identical precompile output (whatever form that actually takes) for many of the functions that get precompiled, must be happening on many users’ machines.
I’ve used ccache [2] before with pretty good results in speeding up compilation locally for large C++ codebases and wondered whether that approach could be taken, but in a distributed way, such that it is likely that the cache entry that you need has already been populated by another user somewhere else in the world. And indeed it looks like there is some work in this direction with ccache [3].
Could Julia use this kind of approach?
[1] TwoHardThings
[2] https://ccache.samba.org/
[3] Ted's Mozilla Blog » Blog Archive » sccache, Mozilla’s distributed compiler cache, now written in Rust