Very slow time to first plot, 2022

Ok, these results you have I believe would match the experience of most people here (maybe some things would be up to two times faster because of beefier desktop computers, but that is not excuse). (Edit: the “2-3 minute” quote in your end paragraph is still weird, but I am focusing on the measures that are explicitly reported in this post. If you can reproduce the “2-3 minute” delays with a new Project file, please share so we can investigate)

With stock julia 1.8 and a package as ridiculously dynamic as Makie you can not do better than the 10-to-20sec you just measured. However, since julia 1.8 it is possible to cache much of this in a sysimage. This is the only way to get millisecond-fast Makie import today. It was actually made possible by improvements to the caching of code in 1.8 and it was not possible in 1.7. Tim Holy is among the heros that made it possible.

In juia 1.9 and more plausibly 1.10, sysimages might not be necessary, thanks to smarter compiled code caching schemes being developed.

There are a couple of good ways to make a sysimage in 1.8. My preferred way is to just make a default (no customizations) sysimage command in VS code. It usually makes good enough sysimages. Other folks prefer AutoSysimage.jl. I completely agree that it is frustrating to have to add these extra steps, but on the other hand, compilation steps like that are normal for compiled languages like Rust and C. Julia is trying to hit a very difficult middle ground between dynamic and compiled-fast, so I am for the moment content with this extra step. As I mentioned, in the next few versions of julia it might not be necessary.

A tangent: I believe it is worthwhile to discuss why this is such a phenomenally big problem in julia. Julia has two very special features other languages do not share: (1) multimethods as the fundamental principle for the entirety of the ecosystem and (2) compiled code. It is very difficult to know what code you need compiled and to not discard the vast majority of already compiled code when importing new libraries that add new methods for pre-existing functions. No one has had to deal with this problem before julia. It is being slowly dealt with. Sysimages basically carry the promise that no significant amount of new methods will be defined, hence they can cache more compiled code (this is very oversimplified borderline misleading explanation).

Edit: I have an experimental simulator for some quantum physics project. It uses DiffEq, QuantumOptics, QuantumClifford, SimJulia, GLMakie, and a bunch of other libraries. Similarly to you I had more than a minute for first-time import even with a stable Project.toml. Before 1.8 I had to be very careful with using Revise and long running julia processes and let blocks to keep my workspace clean of temporary variables. Now I use a sysimage in vscode and import times are less than a second. I had to modify my workflow but I am incredibly productive with this new workflow.

5 Likes