Building an incremental sysimage in seconds instead of minutes

Internals & Design
, , ,
1

Disclaimer: this post was written using AI, but I did my best to make this informative and useful for others. The post has been flagged for being an advertisement, it is not. I hope it passes this time.

This is a research project. I’m putting it out so that others can experiment with it. You have to build both branches from source.

Some background on why I did this. I’m writing a discrete event network simulator in Julia. People write their simulation models on top of it, and not everyone wants to work in the REPL. If someone prefers to run a compiled binary instead, I don’t want to prevent that and lose a customer. The problem was that create_app took two and a half minutes for every source change, which is way too slow to be a real workflow. So I spent some time making the compiler incremental.

The result is a patched Julia (branched off 1.13.0-rc4) and a patched PackageCompiler. Both branches are public and they work today.

What it does

The build result is kept in a store, roughly the way git keeps its objects: every build is a snapshot. When you edit a source file and rebuild, it

  1. boots a Julia process from the previous system image,
  2. diffs the tracked sources and evaluates only the top level expressions that changed,
  3. compiles only what Julia’s own invalidation killed,
  4. links everything else (most of Base, the packages, the rest of your app) from the object code of the earlier builds,
  5. writes a new image and replaces a single file in the bundle, lib/julia/sys.so. The executable, the libraries and the artifacts stay as they are.

No build ever runs your program. The compile roots come from a --trace-compile trace that is taken once, in a throwaway process, when the store is created.

Calls from newly compiled code into reused code are direct calls, resolved by symbol at link time, so there’s no dispatch penalty. A call that isn’t inlined costs 1.0 ns from new code into reused code, and 1.0 ns inside reused code. (The trampoline it replaced cost 43 ns.)

The compiler server

By default the store keeps a compiler process running in the background. It’s started with julia --reactive-server=<socket>, it has the image of the last build loaded, and it stays there between your edits. A rebuild sends the changed expressions to it over a Unix socket, the server applies them and compiles, then forks a child that writes out the image. Because of the fork the parent keeps its heap, its ledger and everything it compiled so far, so your next edit starts from there again.

That’s most of the difference between 0.8 s and 12 s. Without the server every rebuild has to start a process, load the 167 MB image and redo the setup before it can even look at your edit.

The protocol is deliberately boring: a Unix socket, one request per connection, one line in each direction, no libuv. Any build tool can start the server and drive it, one of the gates does exactly that from a Python script that knows nothing but the socket path. status tells you whether the server runs, stop shuts it down, and --no-server does the rebuild in a fresh process instead.

Numbers

Measured on the routing sample of my simulator: a 167 MB system image with 65067 functions, on an AMD Ryzen AI Max+ 395 running Linux, with 8 image threads.

full build (create_app)          155 s     9.7 GB peak
rebuild after an edit            0.8 s     1.3 GB, 37 of 65067 functions recompiled
binary start after a rebuild    0.23 s     same as after a full build

Applying the source diff is 1 to 3 ms of that 0.8 s. The rest is inference of the invalidated cone, precompiling the trace, writing the heap and linking.

The way the image gets written makes a difference. In one test of ten consecutive edits on the same app: overlay 2.1 to 2.6 s, dirty page write 3.1 to 3.3 s, whole relink 7.0 to 7.6 s. The 0.8 s above is overlay mode, measured after I moved the rebuild machinery into the system image itself. With --no-server it’s 12 to 13 s.

How I verified it

The invariant I was after: for any sequence of edits that get applied, the incremental image should be semantically equal to the image you’d get from a full build of the final sources. Semantically equal means the same method tables (same signatures, same lowered code) for every tracked module, a valid code instance for every trace root, the same program output, and no leftover workload state in the heap.

Not byte equal though, and that’s on purpose. Inference recursion limits depend on which root inference started from, so a recomputed code instance can come out different and still be correct.

There’s an oracle that digests an image (method tables, roots, output, globals). The gate scripts create a store, run a chain of edits through it, and compare that digest against a full build of the same final sources. Together the gates cover 14 categories of source change plus the refusals, ten edits through the compiler server, all three image write modes, the trimmed binary, and the command line. One of them also checks that the image of a chain doesn’t grow, and that it keeps exactly one definition per live function.

Two functional checks on top of that. The mean hop count of a simulation goes from 2.308011 to 4.616022 after the edit (exactly double, which is what the edit does), and back when I revert it. And all 14 call shapes I could think of reach the edited code: keyword, varargs, invoke, @cfunction, finalizer, opaque closure, static parameter and so on.

If a change can’t be applied you get a refusal with a reason and a file:line, and the store is left alone. You never end up with a stale image, and the fallback is always a full build.

Command line

julia -m PackageCompiler build  <app_dir> [--package=<dir>] [options]
julia -m PackageCompiler status <app_dir>
julia -m PackageCompiler stop   <app_dir>
julia -m PackageCompiler watch  <app_dir> [--package=<dir>] [options]

build creates the store if there isn’t one in app_dir, otherwise it rebuilds. status prints what’s in the store: when it was created, the snapshots, the overlay chain, whether the server runs, how much the image grew. stop stops the compiler server. watch builds once and then rebuilds every time you save a tracked file, until you hit Ctrl-C.

Options when the store gets created: --package=<dir> is the package to build, --workload=<file> is the script that gets traced for the compile roots, --tracked=<file>=<Module>,... overrides the list of tracked files (by default it’s whatever the package root file includes), --executable=<name>=<main>, and --optimization, --debug-info, --cpu-target for the image itself.

Options for a rebuild:

  • --image=overlay|pages|whole picks how the image gets written. overlay is the default, it writes the new code as a separate shared object and patches the base image word by word. pages rewrites only the dirty pages. whole relinks the whole image.
  • --delta-opt=0..3 is the optimization level for the newly compiled code only. Level 1 is noticeably faster, and the reused code keeps whatever it was built with, so your hot paths stay at -O3.
  • --no-server does the rebuild in a fresh process instead of the resident server. Slower, but nothing stays in memory between builds.
  • --found forces a full build right now.
  • --compact=<n>,<f> does a full build automatically after n saves, or after the image grew by f.
  • --trim also writes a trimmed binary next to the bundle. Your entry point has to be Base.@ccallable julia_main for that.

Every option has an environment variable as its default. On the Julia side everything is a flag, julia --help-hidden lists the --reactive-* ones. From Julia code you can also call create_app(...; reactive = :auto), which turns a build into a rebuild if a store is already there.

How to try it

First build the patched Julia, this is a normal Julia source build:

git clone https://github.com/levy/julia && cd julia
git checkout reactive-compiler
make -j$(nproc)

Then get the client:

git clone https://github.com/levy/package-compiler && cd package-compiler
git checkout reactive

Then, from an environment that has this PackageCompiler:

julia --project=<env> -m PackageCompiler build /tmp/myapp \
      --package=/path/to/MyApp --workload=/path/to/workload.jl
/tmp/myapp/bin/MyApp
# edit something in MyApp/src
julia --project=<env> -m PackageCompiler build /tmp/myapp    # about a second

The design document, the plans with all the measurements and the gate scripts are in contrib/reactive-compiler/ on the Julia branch. If you want to check for yourself that it works, run tool/gate_c.sh. It needs the branch built in usr/ and the PackageCompiler checkout next to it, nothing else.

Limitations

  • Linux only, and one CPU target per store. The image format has no clone tables, and the link needs --gc-sections.
  • A store is tied to the Julia build that created it. If you rebuild Julia, you have to create the store again.
  • Some changes can’t be applied and need a full build: a new dependency, an include of a file that isn’t tracked, a changed module header, a module option, or a type that a method in an untracked file uses.
  • Only your app’s own sources are diffed. If you change a package from the Manifest, that’s a full build.
  • The image format is version 3, so you can’t rebuild from a stock Julia image.