Load time of MTK models

General Usage Performance
, , , ,
1

I am not happy with the TTFX of our MTK models (see: RamAirKites.jl). We already cache our simplified model after building it and use a custom system image. But nevertheless, the time to load a model from our cache is still about 35s, even though a second run in the same REPL takes only 5.5s.

This is what AI explains:

  • ModelingToolkit.build_function doesn’t produce an ordinary Julia method. It produces a RuntimeGeneratedFunction{argnames, cache_tag, context_tag, id, B} — a wrapper whose body (an Expr) is stashed in a Dict (_cache_body, line ~226) that lives inside a dynamically-created module, and whose actual calls go through a @generated function generated_callfunc_body(...) indirection (line 194).
  • The first time that generated function is called with a given id, Julia has to run type inference + LLVM codegen on it, same as any other freshly-seen method — that’s real JIT work, tied to this specific model’s RHS/Jacobian/getter expressions (129 pts, 108 segs, 6 twist groups, 4 winches).
  • Once compiled, the resulting native code sits in the process’s ordinary method-instance cache for the rest of that Julia session. That’s why re-running init! (even on a fresh sam, since the disk deserialize repopulates the same ids) in the same REPL only costs 5.5 s — you’re paying deserialize + object reconstruction, but the generated_callfunc_body methods for those ids are already compiled in this process, so no JIT.
  • A sysimage is built by PackageCompiler walking the precompile caches of the packages passed to create_sysimage (:StableAWEModels, :VortexStepMethod, …) — i.e. methods that exist statically in those packages’ module method tables at using time. The RuntimeGeneratedFunction methods created by eval while test_for_precompile.jl ran during the sysimage build live in a transient module/id created in that build process. They were real, compiled native code — but only inside that now-dead build process. PackageCompiler’s snapshot doesn’t walk “every method that got JIT-compiled while the precompile script executed,” only the target packages’ own precompiled method tables, so this per-model, eval-minted code never makes it into the .so.
    So every fresh julia -J bin/kps-image-1.12.so process starts with an empty method-instance cache for this model’s RuntimeGeneratedFunction ids and has to redo that JIT work once — that’s your 38 s. Inside that same process, it’s free from then on — that’s your 5.5 s. This is a known SciML/ModelingToolkit + PackageCompiler limitation (the whole point of the RuntimeGeneratedFunction indirection is to avoid invalidating a huge sysimage’s method cache every time a user builds a new symbolic model — the tradeoff is that the model-specific code can never be pre-baked into that sysimage).

Is there no workaround to include this compiled code in the system image for production use?