Precompilation and package compiler

Hello, we’re creating binaries using PackageCompiler.jl, and it’s working well, but… right now we have a script create_precompile.jl which we use to create a build/MyPackage_precompile.jl file full of precompile statements, which is git-tracked. The script is run occasionally, as of course the code evolves and the precompile statements is out of date.

This feels bananas to me, because:

  • git-tracking a generated file is generally not recommended
  • Keeping it up to date is a PITA
  • I thought that precompile(f, ...) is equivalent to f(...), except that it does not run the function. Ergo: instead of MyPackage_precompile.jl being full of soon-obsoleted precompilation statement, we could just run the “sample run” code in create_precompile.jl at package compilation time.

I haven’t looked into all of the PackageCompiler details, so maybe my assumptions are way off, but my mental model is that when it runs, it will save to the sysimage all compiled method instances in the current julia process, so it doesn’t matter whether those were created by a sample script, or by precompile statements.

Am I wrong?

After some back-and-forth with colleagues and reading of the docs, it seems the answer is… precompile_execution_file

:tada:

I’m still puzzled why some packages have endless precompile statements, but it seems that we’re good for PackageCompiler

Packages are in general starting to moving over to having a representative workload run during precompilation instead of an explicit precompile file.

1 Like

In addition, you could also use your runtest.jl script to perform the precompilation, so that you do not have code duplication between tests and precompilation. It also forces you to keep test up to date with new features, resulting in overall better testing.

2 Likes

Packages are in general starting to moving over to having a representative workload run during precompilation instead of an explicit precompile file.

@cstjean here’s a related PR with new docs that aims to cover this

1 Like