I’m working on a Julia application where one top-level function has an input called configuration. This function gets called many times with different configurations.
The problem I’m having is that Julia recompiles this big function almost every time that input changes, because there are some internal calls to a package which compile every time, which leads to long first-run times even though the high-level logic is the same. I’ve tried using SnoopCompile to trace the compilation, as well as PrecompileTools and adding explicit precompile calls, but there are so many possible type combinations that it’s difficult to cover them all, and recompilation still happens frequently.
I’m also creating a sysimage where I load the heavy packages I use and where I added some calls in the precompilation_file of the create_sysimage function, but like I said, I don’t think I should put hundreds of different calls in this precompilation file.
I can’t modify much of the internal package code, but I can control how I call it or how I structure the inputs.
What are the best practices to reduce this kind of repeated compilation?
Are there any good strategies for handling functions that depend on many type-specific inputs (e.g. using @nospecialize, caching, or other precompilation approaches)?
Any insight or concrete examples from similar situations would be really appreciated.
Ideally, the configuration parameters should change value, but not change type. The latter triggers recompilation, but the former does not. In general, you want to strive for “type-stable” code where the types are unchanging and statically inferred at compile-time.
What is the type of your configuration object that you are passing to the top-level function?
I’ll add that making inputs vary in value but not type is exactly what makes precompilation caching worth it. If your input types must change often, especially in a large combination (or likely a permutation), then there’s not much point in caching the code for only a small fraction of them. There are several ways you can leverage more precompilation here:
convert the input types in a helper function to a consistent type combination, then feed the converted inputs into a precompiled core algorithm.
force the input types to be more consistent. You don’t have to sacrifice all type-genericity, but you can see how a Vector{T} would be far more constrained than a NamedTuple with arbitrary element types.
refactor with a type like Vector{Any} that doesn’t care about the exact types of the elements, though this is only worth it if you can precompile inner functions over the exact types instead of the much larger set of combinations.
@nospecialize, but this seriously costs performance so it’s suited to special cases.
Of course these are only broad generalizations, I don’t presume that this is enough for your particular problem.