Julia precompilation limits or are there really any?

General Usage
, ,
2

I’m not an expert on the topic, so others more knowledgeable about the compiler can definitely chime in here. But from my understanding:

  • Currently there will always be some compilation cost on the first call. Precompile directives only save type-inferred code, not native code, so the first time a precompiled method is called the compiler still needs to do some work. However, a major part of compiling time is spent in type inference, so precompilation will still significantly cut package loading time in most cases. The reason why native code is not cached is primarily because of method invalidations, which would just force re-compilation and make the point of caching machine code moot. Now that there are tools to diagnose and fix invalidations early on (thanks to the ever-amazing @tim.holy), we might see cache improvement ideas soon!
  • The documentation of SnoopCompile on precompilation limits is referring to the complexity of precompilation. You can’t just slap some precompile directives on your package and expect it to magically solve all your life problems without careful inspection into what precompilation is actually doing in your case. Two examples that text points out: one, if your function requires dynamic dispatch at some point in its body then methods from that point on might have to be resolved during runtime and won’t be cached; and two, even if you pin down the specific types for those methods to precompile, then they still won’t be saved if you don’t own them and your function doesn’t depend on them explicitly in inferred code. For a tutorial on the subject, this recent blog post is an amazing read.
  • @snoopi_deep is a tool to gain some insights into the type inference process. By analyzing where the compiler spends the most time and why, you’ll be able to improve inference, fix invalidations, and prevent unnecessary specializations early on. This will help Julia cache your package much more efficiently and speed up loading time. SnoopCompile provides other tools to generate precompile directives from @snoopi_deep results to help with the process, but again be mindful that this might not significantly improve latency if you don’t take care of the issues mentioned so far.
  • PackageCompiler’s create_sysimage merely saves the current state of your Julia session. Methods are already precompiled according to the package’s precompile directives, the value of PackageCompiler is when you want to save all the subsequent compilation work you depend on in your workflow.
12 Likes