Any way to speed up loading large precompiled packages?

moble
Fri, 12 May 2023 20:37:56 +0000
General Usage

I have a package that my collaborators want to use like a script. Specifically, they want it to load quickly. Once it’s compiled, my part of the code is fast — around 0.01s, compared to the several seconds taken by my equivalent Python package. But loading the code still takes almost 30 seconds, even with all the cool precompilation stuff in Julia 1.9 and PrecompileTools. (Compare this to the 0.15 seconds to load the Python code, and Julia comes out behind.)

The problem is that there are a few packages that my package depends on that just take a really long time to load. I don’t mean to criticize these packages because they are awesome in so many respects, and have so many things going on that I’m sure I couldn’t do any better. But to be specific, they are OrdinaryDiffEq and Symbolics. Even in an otherwise empty project, leaving my own package out of it entirely, and using all the tricks I found in PrecompileTools about making a “Startup” package and “healing” invalidations, it still takes quite a while just to run

using OrdinaryDiffEq

and/or

using Symbolics

(On my 7yo macbook pro, it’s about 22 seconds and 6 seconds, respectively.) There’s not a whole lot of (re)compilation being reported by @time using ..., but the load time is undeniable.

One point seems to be that both of those packages have lots of capabilities I don’t actually use in my package, many of which seem to be part of the precompilation for them (thus leading to huge compilation caches, I think…). Would it be possible and/or useful for me to turn off precompilation of those packages, and just rely on my own precompilation workloads to precompile whatever I do actually use? I’ve tried something like this with the preferences method suggested in PrecompileTools, but it didn’t seem to stop the precompilation from happening. In particular, it didn’t speed up the loading, though I can imagine that’s just because I did something wrong. Is this an avenue I should pursue?

Is there any other option? I had claimed that Julia 1.9 was going to make things so much better for my collaborators (who are all Python users I’ve been trying to convert), but that seems to be falling flat.

36 posts - 13 participants