I’m new here and maybe I’m not doing it the right way, if so I’m sorry for my ignorance.
So I’m interested in precompiling and trimming a Julia program to decrease the startup time and avoid compilation during execution. I’m working on porting a widely used (both in academy and industry) C++ discrete event simulation kernel (omnet++) for communication network simulation and it’s largest model library (INET framework) to Julia. I’m actually one of the core developers of them. I have a number of good reasons to try to make this work in Julia but that’s for another day.
I already have a working simulation kernel and some simpler simulation models already ported. I’m ok with working from the repl and the dynamic compilation works beautifully. The simulations run with speed comparable to c++, sometimes even faster. But it’s not always the case that the user wants to use the repl. So I used the PackageCompiler.jl and juliac (1.13) to get a precompiled and trimmed version of the simulator for a specific model.
I faced several difficulties along the way (like dealing with abstract types which have many concrete implementations). I can already make a precompiled image that is fast when executed, but it’s unnecessarily large and so it takes too much time to initialize. I could also make a trimmed executable (although I had to modify the trimmer for this to avoid comlicating the program) which is small enough (6MB instead of 600MB) but it’s almost 2 times slower then the version running in the repl. Remember, I’m trying to get close to C++ performance here.
I wonder why that would be the case? Isn’t trim supposed to reduce size while keeping performance?
There are many details to this issue and I don’t want to go into them right here. So my question is this: what is the right place to find people who can help me reach my goal?
I MLIR compile cpp thunks AOT then load them with the julia JIT, but julia still has to load and start grabbing thunks to load. No matter what the julia runtime needs to load. Little things like sysimage, AOT, precompile help but julia has to load… JIT is basically delayed compilation.
My simulation kernel dispatches events through a few abstract types, so I patched the compiler to substitute each abstract argument type with the union of its loaded concrete subtypes (a closed world assumption, which trim effectively makes anyway). That made everything resolve, but it also made the optimizer flatten the hot dispatch site in the event loop into enormous isa chains. I measured 144 to 166 KB of generated code per specialization of the loop. That chain, executed once per event, was the whole 2x. The REPL does the same dispatch through the runtime method cache, and that is simply faster than a giant static chain.
The fix, and this is the part I find interesting: use the closed world analysis only for reachability. Above a union split cost threshold the patched optimizer now declines to flatten and leaves the call site dynamic. The concrete target instances (collected from a short warm-up run of the model at build time) are compiled into the image anyway, and the trim verifier accepts a dynamic call when every method it can match has compiled code in the binary. In a closed world that dynamic call can’t reach anything that isn’t there.
Result: a 3.5 MB executable that runs the 3.2 million events in 0.24 s. Startup time is negligible. That’s slightly faster than the REPL on the same model, with bit-identical simulation results at every duration I tested. So trimming now really does only buy size, which is what I hoped for in the first place.
One gotcha for anyone trying something similar: the warm-up run must construct exactly the same types as the real entry point. I had a type parameter differ one level deep (Function where the real engine had a concrete union of action types), the build was clean, and the binary died on the first dispatch with a MissingCodeError.
I’d still love to talk to people about whether something like this sealed world mode could make sense upstream in juliac, and I’m happy to put together a minimal example of the flattening slowdown if that’s useful.
How are you timing the repl run vs the juliac executable? Just curious, your explanation of fixing the slow static chain sounds good so I don’t have anything to add.
I thought so! Only exception I can recall is I/O, maybe it was actually only outputting/printing (something to do with alternative buffering using Core), and a temporary limitation, maybe already lifted?
Great! It would be so nice to have a good trimming compiler besides what the Julia ecosystem already offers.
Actually, we’re considering moving our 20 yo communication network simulation tool from C++ to Julia and not losing fast and small executables are quite high on the priority list. Many users prefer to use their simulations this way.