RepliBuild v3.3.2 — and a plainer re-introduction
The first post in this thread was dense, and rereading it five months later it
buried the part people actually ask me about: why there is an MLIR dialect
inside a binding generator. So this is less a changelog and more a second
attempt at explaining the thing.
What it does
You point it at C or C++ source. It compiles the library itself, then reads the
compiled binary’s debug info to find out what the compiler actually
produced — every struct’s real field offsets, every function’s real signature,
every vtable — and writes a Julia module from that.
The difference from Clang.jl, SWIG or bindgen is one step. They read the headers
and trust them. RepliBuild reads the object file. A header describes intent; the
binary is what your CPU is going to run. Most of the time those agree and it
doesn’t matter. When they don’t — packed structs, bitfields, small structs
passed in registers, anything C++ inheritance touches — a header-based tool
gives you a binding that looks correct and corrupts memory on the third call.
The other half of the design is what happens when it can’t prove a call is
safe. It refuses, at that call site, with a message explaining why, and the
other 5,000 functions in the library still work. I would rather hand you a
module with a hole in it than one that looks complete.
Three ways to make a call
| Tier |
How |
Status |
| 3 |
ccall into the .so |
The default. Every Hub config uses this. |
| 2 |
An MLIR thunk |
The C++ cases ccall can’t express: virtual dispatch, large or packed struct returns, exception-safe calls. |
| 1 |
Base.llvmcall on a per-function bitcode slice |
Experimental, C only, off by default. |
The original post led with Tier 1 as the headline feature. That was optimistic.
It does work — Lua runs 190 of them, and it lets Julia’s JIT inline C into a hot
loop — but it is opt-in, C-only, and I am not calling it production. The tier
that actually carries C++ is Tier 2, which is the rest of this post.
The part nobody expects: MLIR
MLIR normally turns up in compilers and ML frameworks. Here it does something
small and boring, which is exactly why it works.
To call a C++ method from Julia, something has to shuffle the arguments into the
precise shape the C++ ABI expects: this first, small structs split across
registers, large ones passed by a hidden pointer, and a dozen other rules. Every
wrapper tool solves this somehow. The usual answers are to hand-write a C shim,
or to work it out at runtime from a signature string.
RepliBuild writes a tiny program that does the shuffling, and compiles it.
That program is written in a purpose-built MLIR dialect — ops like call this
virtual method through vtable slot 4, marshal this Julia-aligned struct into
its C-packed form — then lowered to LLVM and run. Four things fall out of using
an IR instead of generating C text:
- The offsets in the marshalling code and the offsets in the Julia wrapper are
the same DWARF numbers, read once. They cannot drift apart.
- The ops carry verifiers, so a malformed thunk fails when the module is parsed
rather than when you call it.
- The x86-64 SysV rules live in one readable pass instead of being implied by a
code generator.
- The dialect is written to disk and the JIT registers debug info pointing at
it — so gdb stops inside the generated MLIR by file and line, and
disassemble /s interleaves the dialect ops with the machine code they became.
When a foreign call goes wrong at 2am, that is the thing you want.
No optimization, no graph rewriting, no clever passes. It is a compiler IR doing
the one job that is normally done with string templates.
New in 3.3: those thunks now compile ahead of time
The cost of the above: that MLIR module had to be built and JIT’d in every
process that loaded the wrapper. On a small library nobody notices. llama.cpp
has 3,686 functions, and it was 25 seconds of every single startup.
aot_thunks = true in the TOML compiles the thunks once, at build time, into a
companion .so. Loading the wrapper afterwards is a dlopen.
| llama.cpp wrapper |
JIT |
AOT |
| wrapper load |
24.83s |
5.38s |
| test suite (33 tests) |
35.4s |
1.0s |
| JIT engines spun up at load |
1 |
0 |
The flag had existed for a long time and was switched off in every package,
because it was broken in four ways that only appear at scale — the library I
developed it against has 283 functions and triggered none of them. If you like
that sort of thing the changelog has the full autopsy; the short version is one
regex that exhausted PCRE outright, two separate duplicate-symbol bugs, and an
rpath that let a vendored copy load two copies of a 41 MB library into one
process, which then abort at exit while tearing down each other’s state.
This was the most atrocious bug I have ever dealt with in my life.
That last one is the one worth passing on. The package’s own test suite was
green — 33/33, exit 0 — the entire time, because in the build tree the two paths
happen to name the same file. Only a consumer vendoring the artifacts ever saw
it. A generated wrapper isn’t proven by the tests that ship next to it.
What it looks like
llama.cpp is in the Hub now, and the reference example is a chat client built on
it:
using LlamaChat
list_models() # what's on the box
load("qwen3-coder") # load it and start talking
julia> load("qwen3-coder")
qwen3-coder · 32768 ctx · /help for commands
>>> write a haiku about pointers
…
[9 prompt tok @ 43.2/s · 31 gen tok @ 20.1/s]
>>> /exit
(unloaded)
julia>
Replies render as markdown in the terminal, code blocks and all. /exit frees
the context and the weights before it returns, so the session you come back to
is holding nothing.
That package used to take 32.4 seconds to load. It now takes 0.24 — the
package, that is; the weights still take as long as the weights take.
Where I’m at, honestly
- x86-64 Linux. ABI classification is SysV only. Win64 and AArch64 are not
modeled, and I would rather say that than pretend otherwise.
- C libraries need nothing but Julia — clang comes from a JLL and everything
else runs in-process on Julia’s own libLLVM. C++ and Tier 2 need a system
LLVM + MLIR install. It is by far the largest dependency in the project, and
a C-only library never touches it. check_environment() tells you which tiers
your machine supports.
- Just over twenty libraries in the Hub — lua, sqlite, zlib, cJSON, box2d,
curl, pcre2, imgui, blake3, llama.cpp and the rest. RepliBuild.search("json")
to browse, RepliBuild.use("lua") to get a loaded module.
- Parked, deliberately: Tier 1 llvmcall (works, unproven at library scale)
and the old whole-module LTO payload (it embeds the entire linked module per
call site, which does not survive contact with a real library).
- The known-unbuilt list lives in the repo and stays honest. It is not short.
Holes, edge cases and “this library breaks it” reports are the most useful thing
anyone can send me. If a wrapper refuses a call it shouldn’t, that’s a bug I want
to hear about — the refusals are the design, but a wrong refusal is not.
I rewrote this with claude but I verified everything and most is verbatim from my recent release notes I spent all night making for the 3.3.1 AOT patch, much love.
ABI reconstruction + ABI adaptation compiler between Julia and foreign native code is a more accurate description, think of it as a way to call a cpp function using types extracted from dwarf and giving it to mlir to repack as a thunk that julia can call, rules for that are written once in the dialect and not in the julia wrapper or shimmed into the cpp source, were moving the IR around.
I really need help vendoring my dialect, I really cant figure this part out, Enzyme does it but its under tha same c api version that ships with julias llvm, this needs a seperate mlir version(latest) and i dont want users to have to build or AUR the mlir package for the small dialect its a issue for me.
