I’m working on
MakieLayout.jl so I think a lot about compilation latency. I found the recent discussions about method invalidations quite interesting, as they point to an avenue for package developers to reduce compilation latency. As far as I understand, if you avoid introducing new return types when overloading Base functions, or avoid overloading all together, you can avoid a lot of recompilation.
So I thought about other ways in which the compiler’s work could be reduced. Isn’t it true that a completely self-contained package (doesn’t extend other packages’ or Base’s methods) could be separately compiled and then loaded into a REPL session in binary format? It should not invalidate any previous functions, so maybe it could just be “added” to the currently compiled code. I wonder if a plotting package like
Makie.jl is a good package to try such an approach, because it doesn’t have any “algorithms” that need to be compiled through and through for custom types. Rather after the surface layer, all complex types will be converted to arrays of Floats or similarly basic stuff anyway.
So if we have a package that tries hard not to overload or extend existing functions, could we then not ship a completely precompiled (as in binary, not just type inferred) version of it which can be immediately loaded, without any more precompilation being necessary? We’d still have all the Julia magic at the surface level and be able to put any kind of types into
plot!(), but all the backend code would be compiled.