# Cross-language LTO with Base.llvmcall: getting past the "IR at every callsite" wall

**URL:** <https://discourse.julialang.org/t/cross-language-lto-with-base-llvmcall-getting-past-the-ir-at-every-callsite-wall/138429>\
**Category:** Performance\
**Created:** [July 23, 2026, 1:48am UTC](https://discourse.julialang.org/t/cross-language-lto-with-base-llvmcall-getting-past-the-ir-at-every-callsite-wall/138429 "2026-07-23T01:48:59Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![obsidianjulua](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/obsidianjulua/32/221495_2.png) [@obsidianjulua](https://discourse.julialang.org/u/obsidianjulua)\
**Post date:** [July 23, 2026, 1:48am UTC](https://discourse.julialang.org/t/cross-language-lto-with-base-llvmcall-getting-past-the-ir-at-every-callsite-wall/138429/1 "2026-07-23T01:48:59Z")

</div>

I’ve been parked hard on `Base.llvmcall` for a while and I can’t leave it alone. At first I was cruising — _“oh man, this replaces everything; I’ll just inline every FFI call and let Julia LTO the whole thing.”_ Then I tried it at scale and the dream broke: `llvmcall` embeds the **entire IR string literal at every callsite**. Wall of text, blown-up code size, done. I shelved it — but I’ve got an idea. Hear me out.

**The dream:** embed C LLVM bitcode into the wrapper and let Julia’s own codegen inline it — cross-language LTO, at compile time. Two snags:

1. `llvmcall` needs its IR as a **compile-time-constant** literal (the `(module_ir, entry_name)` string-tuple form). A `const` you fill in ` __init__ ` is invisible to codegen, so that route’s out.
2. You don’t want the library’s whole optimized module — or a wall of IR — sitting at every callsite (this is the wall that parked me because julia crashes on the size of the IR and it looks ugly even if julia can embed it all).

**The reality:** _slice_ the library’s optimized bitcode down to one function — its definition plus `declare`s for what it references — into a sister `.ll`, then pull it in with `@generated` so the wrapper _source_ stays clean while `llvmcall` still gets its literal:

```julia
@generated function xxh64(p::Ptr{UInt8}, n::Csize_t)
    ir = read(joinpath(@ __DIR__ , "slices", "xxh64.ll"), String) # jargon lives here
    :(Base.llvmcall(($ir, "XXH64"), UInt64, Tuple{Ptr{UInt8},Csize_t}, p, n))
end

```

Now `XXH64`’s actual instructions get inlined into the caller and optimized as one unit — the C isn’t a call anymore, it’s just IR. The literal blowup is gone because each wrapper carries only its own one-function slice, not the whole module × every callsite.

---

<div class="post-metadata">

**Author:** ![obsidianjulua](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/obsidianjulua/32/221495_2.png) [@obsidianjulua](https://discourse.julialang.org/u/obsidianjulua)\
**Post date:** [July 31, 2026, 11:59pm UTC](https://discourse.julialang.org/t/cross-language-lto-with-base-llvmcall-getting-past-the-ir-at-every-callsite-wall/138429/2 "2026-07-31T23:59:47Z")

</div>

Follow-up to my post from a couple weeks back. The `@generated` route is wired into my generator now and running at scale, so here’s the report: how it works, real numbers, and one landmine I hit that I haven’t seen documented anywhere — **executing llvmcall’d IR inside a precompile worker can deadlock it permanently**. If you only read one section, read that one.

## Where it landed

The slicing + `@generated` combo from the last post is now what my wrapper generator emits. Live on Lua 5.4.7: 209 functions sliced, 190 call sites emitted across 189 functions, and the wrapper passes the full deep test (callbacks, coroutines, `luaL_error` longjmp across a Julia frame, bytecode roundtrip, GC stress). `code_typed` on a wrapped function shows the C body spliced straight into the caller — the C isn’t a call anymore, it’s just IR, and LICM will happily hoist it out of your loop like it was Julia code all along.

But the emitted shape grew a brain since the sketch in my first post, because the naive version has a failure mode that will eat someone’s afternoon.

## The landmine: llvmcall + precompilation = a silent, permanent deadlock

Here’s a minimal package. Precompiling it on 1.12.6 never finishes — no error, no timeout, the worker just parks at 0% CPU forever:

```julia
module Wedge
using Libdl
const SO = "/path/to/liblua.so" # any C library you dlopen at runtime
const IR = """
declare i32 @lua_gettop(ptr)
define i32 @probe(ptr %L) alwaysinline {
  %r = call i32 @lua_gettop(ptr %L)
  ret i32 %r
}"""
probe(L) = Base.llvmcall((IR, "probe"), Cint, Tuple{Ptr{Cvoid}}, L)

if ccall(:jl_generating_output, Cint, ()) == 1
    dlopen(SO, RTLD_LAZY | RTLD_GLOBAL) # doesn't help!
    L = ccall((:luaL_newstate, SO), Ptr{Cvoid}, ())
    probe(L) # ← worker parks here, forever
end
end

```

Stack of the wedged worker: `pthread_mutex_lock` ← `jl_unique_gcsafe_lock` ← `jl_emit_codeinst_to_jit_impl`, single thread, nobody visibly holding the lock. I bisected the boundary:

- llvmcall with **no** external `declare`s during precompile: fine.
- llvmcall declaring a **process-image** symbol (libc `cos`): fine.
- llvmcall declaring a symbol from a **dlopened library** : permanent deadlock. dlopening the library inside the worker first changes nothing.

Two extra twists that make it nastier than it looks:

1. **You don’t have to call it.** A top-level branch that is _never taken_ — `if jl_generating_output() && some_false_thing; probe(L); end` — still wedges, because the toplevel thunk gets compiled and inference reaches the llvmcall and emits it anyway.
2. It’s **not specific to `@generated`**. The plain `const IR` + ordinary function shape from my first post hits it identically. Any package that ships this pattern is one PrecompileTools workload (or one `precompile(f, ...)` directive) away from hanging its consumers’ builds.

## The fix: let the generator decide, at generation time

This is where `@generated` stops being a convenience and becomes load-bearing. The generator body _runs in the process that’s compiling_ — so it can ask “am I a precompile worker?” and splice a **different body**. A runtime `if` can’t do this (inference reaches both arms — see twist #1). A `const` can’t do this. Only a generated function gets to make a per-process, per-context decision about what the method _is_:

```julia
const _SLICE_lua_gettop = joinpath(@ __DIR__ , "slices", "lua_gettop.ll")
isfile(_SLICE_lua_gettop) && include_dependency(_SLICE_lua_gettop)

@generated function _TIER1_lua_gettop(__ptr_L::Ptr{lua_State})
    # llvmcall is opportunistic: any doubt resolves to the ccall body.
    if ccall(:jl_generating_output, Cint, ()) == 1 || !isfile(_SLICE_lua_gettop)
        return :(ccall((:lua_gettop, LIBRARY_PATH), Cint, (Ptr{lua_State},), __ptr_L))
    end
    ir = read(_SLICE_lua_gettop, String)
    return :(Base.llvmcall(($ir, "lua_gettop"), Cint, Tuple{Ptr{lua_State}}, __ptr_L))
end

function lua_gettop(L::Any)::Cint
    __cc_L = Base.cconvert(Ptr{lua_State}, L)
    __ptr_L = Base.unsafe_convert(Ptr{lua_State},__cc_L)
    GC.@preserve __cc_L begin
        return _TIER1_lua_gettop(__ptr_L)
    end
end

```

Inside a precompile worker the kernel _is_ a ccall — precompiles clean, workloads run, correct results. At runtime the generator runs again and the kernel _is_ the llvmcall. I verified both directions: the consumer package that used to deadlock now precompiles to completion, and `code_typed` in a fresh session shows llvmcall. (If a precompiled specialization does get cached from a workload, you keep the ccall body for that signature — correct, just not inlined. Fine by me: my design rule is that llvmcall is a passenger, never the driver. Anything ambiguous demotes to ccall and the wrapper keeps working.)

The `isfile` guard is the same philosophy: ship or relocate the wrapper without its `slices/` directory and it degrades to a plain ccall wrapper instead of refusing to load.

## What `@generated` buys over the `const` shape, concretely

- **No load-time I/O.** 190 slices used to mean 190 `read()`s at `using` and ~1.5 MB of strings resident whether you called anything or not. Now: zero until first call, and only for functions you actually use.
- **No double storage.** `const IR = read(...)` serializes the IR string into the `.ji` _on top of_ the `.ll` files you ship. The generated version stores paths.
- **Staleness still works.** `include_dependency` is content-tracked on 1.11+, so editing a slice still invalidates the cache; the read living in the generator doesn’t change that.
- **The precompile deadlock becomes unreachable by construction** — which, given twist #1 above, I don’t know how to guarantee any other way.

## Sharp edges checklist (things that cost me a debugging session each)

- **An unresolved `declare` does not error.** ORC prints “Symbols not found” and blocks forever on first call. Pre-flight every declared symbol with `dlsym` against the library handle _before_ you ship a slice, and demote to ccall on a miss.
- **Mirror ccall’s GC discipline exactly** : `cconvert` → `unsafe_convert` → `GC.@preserve` around the call. And `Ref{T}` is `ptr addrspace(10)` to llvmcall while C IR wants plain `ptr` — convert to `Ptr{T}` explicitly.
- **Key your slice files and consts on the mangled symbol, never the pretty name.** Two C symbols can collapse to one Julia name, and because llvmcall resolves at codegen, the loser fails on first call, long after wrap time.
- **Don’t embed internal constants unless they’re `unnamed_addr`.** The `.so` keeps its copy and the JIT gets a second one at a different address — any address-identity check silently diverges between your inlined calls and your ccall calls. Mutable statics: never embed, bind by symbol.

Everything here is generated by [RepliBuild.jl](https://github.com/obsidianjulua/RepliBuild.jl) — slicing lives in `src/IRGen/Slicer.jl`, the kernel emission in `src/Wrapper/C/GeneratorC.jl`. The deadlock repro is small and self-contained if anyone from the compiler side wants it as an issue — arguably `jl_emit_codeinst_to_jit` should error there, not hang.

---

<div class="post-metadata">

**Author:** ![adienes](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/adienes/32/37459_2.png) [@adienes](https://discourse.julialang.org/u/adienes)\
**Post date:** [August 1, 2026, 1:13am UTC](https://discourse.julialang.org/t/cross-language-lto-with-base-llvmcall-getting-past-the-ir-at-every-callsite-wall/138429/3 "2026-08-01T01:13:38Z")

</div>

please contribute in your own words, and don’t just paste AI text.

---

<div class="post-metadata">

**Author:** ![obsidianjulua](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/obsidianjulua/32/221495_2.png) [@obsidianjulua](https://discourse.julialang.org/u/obsidianjulua)\
**Post date:** [August 1, 2026, 3:31am UTC](https://discourse.julialang.org/t/cross-language-lto-with-base-llvmcall-getting-past-the-ir-at-every-callsite-wall/138429/4 "2026-08-01T03:31:00Z")

</div>

Its mine but refined, I wouldn’t make it look this nice for markdown.

---

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [August 1, 2026, 1:44pm UTC](https://discourse.julialang.org/t/cross-language-lto-with-base-llvmcall-getting-past-the-ir-at-every-callsite-wall/138429/5 "2026-08-01T13:44:15Z")

</div>

It sure does seem to have done a lot more _load bearing_ work than that 😛

---

<div class="post-metadata">

**Author:** ![MDSW](https://avatars.discourse-cdn.com/v4/letter/m/94ad74/32.png) [@MDSW](https://discourse.julialang.org/u/MDSW)\
**Post date:** [August 1, 2026, 2:15pm UTC](https://discourse.julialang.org/t/cross-language-lto-with-base-llvmcall-getting-past-the-ir-at-every-callsite-wall/138429/6 "2026-08-01T14:15:26Z")

</div>

That’s just the _shape_ of things these days. 🤣

---

<div class="post-metadata">

**Author:** ![obsidianjulua](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/obsidianjulua/32/221495_2.png) [@obsidianjulua](https://discourse.julialang.org/u/obsidianjulua)\
**Post date:** [August 1, 2026, 6:11pm UTC](https://discourse.julialang.org/t/cross-language-lto-with-base-llvmcall-getting-past-the-ir-at-every-callsite-wall/138429/7 "2026-08-01T18:11:03Z")

</div>

> [@Mason](#):
>
> It sure does seem to have done a lot more _load bearing_ work than that 😛

Yeah Im not hiding any of that at all, Im no top flight engineer but Fable 5 was able to unblock alot of this for me, I had the CONST all wrong and how I wanted to slice each function and its deps into another .ll that the function keeps with it, now it looks really nice with no IR string embedded. Its lonely in the julia space so building with AI just seems natural now.

---

<div class="post-metadata">

**Author:** ![adienes](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/adienes/32/37459_2.png) [@adienes](https://discourse.julialang.org/u/adienes)\
**Post date:** [August 1, 2026, 6:22pm UTC](https://discourse.julialang.org/t/cross-language-lto-with-base-llvmcall-getting-past-the-ir-at-every-callsite-wall/138429/8 "2026-08-01T18:22:14Z")

</div>

there is no problem at all building with AI, reviewing code with AI, etc.

the only thing I am challenging is pasting walls of AI-generated prose into the forum. it’s much nicer to interact with humans

---

<div class="post-metadata">

**Author:** ![obsidianjulua](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/obsidianjulua/32/221495_2.png) [@obsidianjulua](https://discourse.julialang.org/u/obsidianjulua)\
**Post date:** [August 1, 2026, 9:29pm UTC](https://discourse.julialang.org/t/cross-language-lto-with-base-llvmcall-getting-past-the-ir-at-every-callsite-wall/138429/9 "2026-08-01T21:29:43Z")

</div>

I get what your saying, If you as a person want to spend time to read something then it should be from another person. That person should take the time to learn what was generated by an LLM and present a readable explanation. I agree, regardless if an LLM generated the code you should be able to understand it and argue its functionality. Imma make a point to write updates personally and mark what part the LLM generated that I had to read to understand rather than write myself or what was optimized by AI and why, I do post the blocks sometimes because I do honestly feel like the AI explains it better for the overall better than I would for each case because even though I understand the endpoints I dont always understand the insane regex stuff fable 5 cruises through like its nothing.