Joovy - Faster compilation

Tooling
1

Hey,

Paul here, the guy behind Flexible Julia. So there’s been a couple of posts regarding Julia performance that have been bugging me (in particular this one: julia-debugging-is-extremely-slow) by @tue. So i started to mess with different ideas on how to improve it further. And being a Java guy, I thought, maybe we just miss something like Groovy for Java (only for Julia), ie. a compiler that optimizes when something is being compiled.

What’s Joovy.jl:
I built a compilation layer that makes Julia feel faster during development by not compiling everything at full optimization upfront. It is called Joovy and you can find it here: github.com/ilscipio/joovy

Explainer:
Julia’s compiler is thorough. Every function you touch gets fully optimized before you can use it. That is great for production code but painful during development. Precompilation is also incredibly slow, since you compile all dependencies in full. Regardless of what is needed to execute.

So Joovy is like an addon to the compiler. Instead of compiling everything at full optimization, Joovy uses three tiers. Tier 0 is basically not-compiled. Fast to load, but slow to execute. Tier 2 is native Julia. Tier 1 is the middle ground. It does lazy module loading. When you open a file, Joovy parses it and builds a dependency graph but compiles nothing. When you actually call a function, it compiles that function and its dependencies. Everything else stays untouched. External packages get the same treatment through configurable dev-mode tiering. Functions start at tier 1. If you call them enough, they get promoted to native speed (tier 2) in the background. You never wait for the full compile on first use.

So this is not Revise.jl or Julia compiler, it sits in between as a way to optimize how something is processed. (Revise compiles and hot-swaps on existing code, native compiler handles the full packages, Joovy optimizes when compilation happens).

Experimental

This is all experimental at this point. I built it primarily for Flexible Julia so the IDE can handle compilation more intelligently behind the scenes. And you can turn it on there in the settings (requires a repl restart). But the package works standalone too. Since this is all new, it is of course possible I messed up. Maybe my own experiment result are misleading, maybe it is not working everywhere. Who knows - but this should improve over time.

If you are curious, check it out: GitHub - ilscipio/joovy: Dynamic compilation engine for Julia. Tiered optimization, lazy modules, and hot-swapping at runtime. · GitHub

Feedback and bugfixes welcome. If anyone has thoughts on the approach or wants to poke holes in the design, I am all ears. Obviously, this is experimental for a reason at this point…

Oh and: this is open source. So anyone who wants to contribute is well invited to do so. Fork, expand, create PRs or do whatever.

2

That’s… amazing? What’s the catch?

I don’t want to sound too negative, the question is motivated by my not having enough of a technical understanding to get how this works, and the fact that something like this has been discussed in the community for as long as I can remember (and I stayed using Julia in 0.3 days!), so the fact that it hadn’t happened let me to believe that it was a herculean task requiring the concerted long term efforts of many of the core devs.

Is this something that everyone should be using all the time or are there drawbacks to the approach that make it unsuitable for some situations

3

It is definitely not something you should be using “all the time”. It is something you should be using during development and very much experimental.

Since you asked for it, here’s the caveat: it is “good enough” to use when you are developing, because waiting for something to process while you are editing is probably better than living with a long compilation process. (I really did have Groovy in mind with this) But during production use, the native compiler is probably better because things are always fast at startup. (and you only deploy once).

It is also “not great” when you use it without the help of an ide that can auto wrap calls and handle the syntax for you. It is also only a first shot, my own experiments were limited to my own code. So expact the code to be unoptimized yet. We can surely do better over time.

4

A comparison:

joovy:


[ Info: FlexibleIPC: Trying 1 host(s) on port 52342: 127.0.0.1
[ Info: FlexibleIPC: Connected to IDE on 127.0.0.1:52342
[ Flexible Julia IDE integration active ]

┌ Plugin-injected modules in Main:
│   FlexibleIPC
│   FlexibleJuliaDisplay
│   VSCodeServer
├ Plugin-set environment variables:
│   GKSwstype = 100
│   MPLBACKEND = Agg
│   JULIA_IDE_DISPLAY = 1
│   GMT_SESSION_TYPE = extern
│   DISPLAY_IN_VSC = true
│   JULIA_IDE_AUTO_JOOVY = 1
│   JULIA_IDE_SVG_PLOTS = 1
│   JULIA_IDE_SHOW_INJECTIONS = 1
├ Display stack:
│   [1] TextDisplay
│   [2] Main.FlexibleJuliaDisplay.IDEDisplay
└ Hook version: 2026.9.5

Precompiling Joovy finished.
  1 dependency successfully precompiled in 2 seconds
[ Info: Joovy loaded - dev mode active (tier 1)
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.12.3 (2025-12-15)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org release
|__/                   |

julia> println("# Executing 149 lines: # Joovy Compilation Benchmark # # Open this file in the IDE,..."); include("C:/Users/pp/AppData/Local/Temp/julia_repl_11564966733672911642.jl")
# Executing 149 lines: # Joovy Compilation Benchmark # # Open this file in the IDE,...

======================================================================
  JOOVY COMPILATION BENCHMARK
  Joovy detected: YES
======================================================================

──────────────────────────────────────────────────────────────────────
  PASS 1 — First call (includes compilation)
──────────────────────────────────────────────────────────────────────
  Running: first call
    make_scatter            1.831 s
    make_heatmap            0.175 s
    make_histogram          0.258 s
    make_dataframe          0.046 s
    make_surface            0.085 s
    make_contour            0.147 s
    make_bar_chart          0.203 s
    make_line_plot          0.158 s
    full_report             0.311 s
    heavy_report            0.478 s
    TOTAL                   3.690 s

──────────────────────────────────────────────────────────────────────
  PASS 2 — Second call (already compiled)
──────────────────────────────────────────────────────────────────────
  Running: second call
    make_scatter            0.000 s
    make_heatmap            0.000 s
    make_histogram          0.001 s
    make_dataframe          0.000 s
    make_surface            0.000 s
    make_contour            0.000 s
    make_bar_chart          0.001 s
    make_line_plot          0.000 s
    full_report             0.002 s
    heavy_report            0.004 s
    TOTAL                   0.010 s

======================================================================
  SUMMARY
======================================================================
  Function                1st (s)    2nd (s)    Speedup
  ------------------------------------------------------
  make_scatter              1.831      0.000    3665.6x
  make_heatmap              0.175      0.000     466.0x
  make_histogram            0.258      0.001     299.2x
  make_dataframe            0.046      0.000    1704.3x
  make_surface              0.085      0.000     273.5x
  make_contour              0.147      0.000     531.5x
  make_bar_chart            0.203      0.001     361.6x
  make_line_plot            0.158      0.000     439.0x
  full_report               0.311      0.002     132.9x
  heavy_report              0.478      0.004     115.1x
  ------------------------------------------------------
  TOTAL                     3.690      0.010     378.0x

  Mode: JOOVY (lazy tier-1 compile on first call)
  Tip:  Run once with Joovy ON, restart REPL with Joovy OFF, run again.
        Compare the 1st-call totals to see lazy vs eager compile cost.
======================================================================

(first_call = [1.83061, 0.174924, 0.257977, 0.045505, 0.0850444, 0.1467, 0.202772, 0.157716, 0.310984, 0.477975], second_call = [0.0004994, 0.0003754, 0.0008621, 2.67e-5, 0.0003109, 0.000276, 0.0005608, 0.0003593, 0.0023396, 0.0041526], names = ["m
ake_scatter", "make_heatmap", "make_histogram", "make_dataframe", "make_surface", "make_contour", "make_bar_chart", "make_line_plot", "full_report", "heavy_report"], total_1st = 3.69021, total_2nd = 0.0097628, joovy = true)

julia>

Native:

[ Info: FlexibleIPC: Trying 1 host(s) on port 56168: 127.0.0.1
[ Info: FlexibleIPC: Connected to IDE on 127.0.0.1:56168
[ Flexible Julia IDE integration active ]

┌ Plugin-injected modules in Main:
│   FlexibleIPC
│   FlexibleJuliaDisplay
│   VSCodeServer
├ Plugin-set environment variables:
│   GKSwstype = 100
│   MPLBACKEND = Agg
│   JULIA_IDE_DISPLAY = 1
│   GMT_SESSION_TYPE = extern
│   DISPLAY_IN_VSC = true
│   JULIA_IDE_SVG_PLOTS = 1
│   JULIA_IDE_SHOW_INJECTIONS = 1
├ Display stack:
│   [1] TextDisplay
│   [2] Main.FlexibleJuliaDisplay.IDEDisplay
└ Hook version: 2026.9.5

               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.12.3 (2025-12-15)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org release
|__/                   |

julia> println("# Executing 149 lines: # Joovy Compilation Benchmark # # Open this file in the IDE,..."); include("C:/Users/pp/AppData/Local/Temp/julia_repl_16242062654936921354.jl")
# Executing 149 lines: # Joovy Compilation Benchmark # # Open this file in the IDE,...
Precompiling Plots finished.
  7 dependencies successfully precompiled in 96 seconds. 172 already precompiled.
Precompiling DataFrames finished.
  1 dependency successfully precompiled in 60 seconds

======================================================================
  JOOVY COMPILATION BENCHMARK
  Joovy detected: NO
======================================================================

──────────────────────────────────────────────────────────────────────
  PASS 1 — First call (includes compilation)
──────────────────────────────────────────────────────────────────────
  Running: first call
    make_scatter            2.055 s
    make_heatmap            0.117 s
    make_histogram          0.253 s
    make_dataframe          0.042 s
    make_surface            0.080 s
    make_contour            0.136 s
    make_bar_chart          0.209 s
    make_line_plot          0.131 s
    full_report             0.313 s
    heavy_report            0.188 s
    TOTAL                   3.524 s

──────────────────────────────────────────────────────────────────────
  PASS 2 — Second call (already compiled)
──────────────────────────────────────────────────────────────────────
  Running: second call
    make_scatter            0.000 s
    make_heatmap            0.001 s
    make_histogram          0.001 s
    make_dataframe          0.000 s
    make_surface            0.000 s
    make_contour            0.000 s
    make_bar_chart          0.001 s
    make_line_plot          0.001 s
    full_report             0.002 s
    heavy_report            0.004 s
    TOTAL                   0.010 s

======================================================================
  SUMMARY
======================================================================
  Function                1st (s)    2nd (s)    Speedup
  ------------------------------------------------------
  make_scatter              2.055      0.000    4761.6x
  make_heatmap              0.117      0.001     209.7x
  make_histogram            0.253      0.001     246.9x
  make_dataframe            0.042      0.000    1410.6x
  make_surface              0.080      0.000     169.9x
  make_contour              0.136      0.000     326.6x
  make_bar_chart            0.209      0.001     370.4x
  make_line_plot            0.131      0.001     251.6x
  full_report               0.313      0.002     145.7x
  heavy_report              0.188      0.004      48.5x
  ------------------------------------------------------
  TOTAL                     3.524      0.010     351.1x

  Mode: STANDARD (eager full compile on include)
  Tip:  Run once with Joovy ON, restart REPL with Joovy OFF, run again.
        Compare the 1st-call totals to see lazy vs eager compile cost.
======================================================================

(first_call = [2.05464, 0.117376, 0.252827, 0.0416138, 0.0801208, 0.135577, 0.209271, 0.131185, 0.313229, 0.187727], second_call = [0.0004315, 0.0005597, 0.0010239, 2.95e-5, 0.0004716, 0.0004151, 0.000565, 0.0005215, 0.0021502, 0.0038681], names =
["make_scatter", "make_heatmap", "make_histogram", "make_dataframe", "make_surface", "make_contour", "make_bar_chart", "make_line_plot", "full_report", "heavy_report"], total_1st = 3.52357, total_2nd = 0.0100361, joovy = false)

Note: precompilation time is almost gone at the cost of a slightly slower execution time. I need to check if this was still related to my joovy caches, so please feel free to run your own tests.

5

Nice work! This sounds similar to this attempt in the Julia repo: [experimental] codegen: add tiered compilation - Pull Request #61888 - JuliaLang/julia - GitHub. I’m not sure if the implementation is, but reading the comments in that PR results don’t seem currently very promising, but there is still activity so still to see I think. Is the approach the same as Joovy there? I expect that something like this is very tied to the Julia compiler, and would benefit from an implementation in the core repo

6

Oh wow. I did not know about that at all. I skipped over it just now (only the first post) and looks very similar indeed from the basic concept. I guess if it started from a similar idea as mine, then ending up with a tiered approach feels natural. So not surprised. But that thread is interesting and I should check it out later!

Thanks for this!

7

Nice!

I think you’re not doing on-stack-replacement, right?

On-stack-replacement is the following mechanism: If we end up in a long-running function (e.g. big loop), it might become worthwhile after some time to upgrade the optimization tier of the function. E.g. the function contains the main code of a large analysis, and is supposed to run for 48 hours.

So the “cold code”-tiers contain checkpoints within loops, and eventually pauses the function, compile it, and goes back.

The simple but annoying technical problem is that you need to transplant the current runtime state from the old unoptimized code into the newly compiled optimized code. A simple variant is: The new compiled function for the on-stack-replacement is only executed once, and can take all the current state into account.

For general availability tiered compilation, I fear that the really difficult issue is around inlining and constant-propagation interacting with on-stack-replacement.

The general goal should be: If you run code with “everything compiled before” vs “tiered compilation”, then we only want O(1) overhead in the worst-case.

In other words: We want performance to eventually catch up to “everything compiled before” mode; and that means deterministically reaching the exact same inlining / const-prop decisions. That’s difficult!

But the world without reaching the deterministic same inlining decisions is a world that looks like the java virtual machine: You cannot really reason about the machine-code for methods, because too many decisions depend on what code ran before. Keyword “profile pollution”.

I understand why JVM accepts that downside to their implementation of tiered compilation, and it’s probably the right choice for them. But I don’t think it would be the right choice for julia. So we have a much higher bar to meet, with much less resources :frowning:

We might get away with some annotations like @long_and_hot function ... that prevent tiered compilation, though.

8

Yeah, I am not really doing on-stack replacement.

Our internals simply reeval the original sources and replace the method when it is possible. Those would come into effect the next call, not during. So it is recompile and swap for the next call, which is probably okay during development.

I had a look at that other discussion by now, btw, and I think one of the biggest differences between the julia lang discussion and mine is that I am not aiming for a faster compiler, I am aiming for a better behavior during development. So many of the things that would be a big concern for a running system are not really big ones for me.

I should say that your concern about a JVM-style situation doesn’t really apply to what I’m doing.
I’m not doing speculative or profile-guided compilation. Nothing about what ran before feeds into the generated code. A promotion just takes the original source and recompiles it at a higher optimization level. So the machine code for a promoted method is the same as if I had defined it normally at that level. There’s no profile pollution, because there’s no profile.

For a leaf function that’s exactly the ahead-of-time result. For a function that calls other functions that are still cold, the inlining can differ for a while - but once it gets promoted, Julia’s own invalidation kicks in and recompiles the callers from source. So it converges to the same decisions you’d get from compiling everything up front. It just gets there a bit later.

The price we pay is extra recompilation, not unpredictable machine code. And since I only care about the development loop, a few extra recompiles are fine. I don’t need to care about the extra overhead - I just need the REPL to feel instant and then quietly catch up to native.

What you are describing is of course true, though, but I think I can get away with it a little nicer, because my scope is narrower.

That said, I do think your suggestion about an annotation or something to mark functions that should be left untouched does make sense. You want to be able to mark code that won’t get upgraded by joovy because it may be taking away compute time whilst getting stuck in a loop. Likewise, I may want to check available resources while recompiling - perhpas something to work on next…

9

Btw, are you Ian from that other discussion?

If so, I’d love to get your opinion on my approach (and perhaps even check out the sources). I know it isn’t the golden compiler we dream of, but it could turn into an immediate improvement :slight_smile:

10

Not me, sorry, that would be @ianshmean

11

Just being curious, would Makie.jl a good place for this? Looks like they want these staff to reduce TTFX a lot?

12

Depends. Do you mean for them to implement?

Not entirely sure they can (but i could be wrong). Because this is more of a compilation issue and how you approach it. So at least to me it becomes a matter of placing it either in the tooling or the core Julia compiler. For various reasons the Julia compiler is more complex than handling it in the tooling (as the other GitHub thread shows) and both should try to optimize for different situations.

I’d argue that it makes sense to do it in IDEs etc., because there you have the freedom of just making harsh cuts. A compiler probably has a harder time. When something like Makie asks “how do we reduce TTFX,” they are limited by what the compiler gives them. They can add precompile tools, use @nospecialize, or restructure internals - but that’s all working within the compiler’s rules. And those rules are necessarily conservative, because the compiler serves everyone: production, CI, development, batch scripts.

Tooling sits on top and can be opinionated. There you can easily say: “don’t bother optimizing this function, compile it as fast as possible, if it gets called 10 times we’ll promote it.” A compiler can’t really do that.

That said - Makie is one of the packages I benchmark against. It is huge and takes forever to precompile. I already prepared a quick update on my internal tests and improvements, that I will post right after answering :slight_smile:

13

So, just pushed an update. I focused on a few sets of precompilation updates for this.

I did some work on the cold start path (no caches, loading everything from source) so we can see if we can really bring precompilation times down and compared it to native julia precompilation.

CairoMakie benchmark on Julia 1.12.3:

Scenario Time
Native cold (precompile all) ~330s
Joovy cold ~173s
Native warm (from cache) ~10s
Joovy warm ~10s

So joovy gets a 2.x faster cold start. Warm is identical.

What it does: it skips precompilation (via --compiled-modules=existing), then before any using runs, sets all loaded modules to low optimization via @compiler_options and hooks into Base.package_callbacks to tier each dependency as it loads. Also sets max_methods=1 during loading so the compiler doesn’t waste time on union splitting. After everything is loaded, packages get promoted back to native in the background.

I tried a few things that didn’t work: --optimize=0 globally, --compile=min (interpreter is slower than just compiling at O0 for heavy packages), and a few other things. Most “obvious” compiler flags either break something or optimize the wrong thing.

Anyway, so the result is “good” for precompilation now, but 173s is still slow. But hey, things are improving. If anyone else has any good idea on this, let me know - happy to try it out.

14

How does a reduction from 330s to 173s make it 2.4 times faster? With 165s you would “only” be twice as fast.

15

My mistake, you are right. ~2 times :slight_smile:

But like I said - it is baby steps right now. Anything that trims it down during the actual developer experience is good in my book.

16

So for someone like me (statistician running models blah blah blah and not development or production), how would you see this benefiting me?

17

It depends on how you work and what you do with it.

If most of your time is spent running a very expensive simulation and the compilation and build times are a non-issue - leave it off.

If you are still developing the code, having to constantly redeploy, maybe even in uncached runtime environments, clean Jupyter/Pluto/REPL sessions, then you may benefit from it, because it should (at least in theory) move the heavy lifting down towards when you are running the code. Not upfront. And ideally also compile less of it - only the functions you actually call in your projects, only dependencies you really need.

You do have other options of course, too: hot swapping via revise.jl, using our debugger or so. All of those are also accessible via the plugin I develop. But there’s limits to that. And this makes it less painful to those that really just need to get going.

Bonus answer: if you follow the discussion that came from first-time julia users, then it is also quite clear that to them the added wait-time of compilation/precompilation is a real pain-point. This helps ease them in.

18

So, just pushed an update.

I had this idea that perhaps we can start precompilation as you type and do it in the background. That way your packages are compiled before you run your program. So I added that as a new option (Settings - Languages & Frameworks - Julia - Experimental - “Enable Joovy lazy compilation”). Afterwards, the IDE watches your using/import lines while you edit and precompiles those packages in a background process - shows up like indexing, cancellable. On my machine adding “using CairoMakie” and running went from a minute long wait to ~5s, simply because by the time i ended up in repl it had already compiled.

Sessions also record what they actually JIT (–trace-compile) and replay it into a small per-project warmup package that auto-loads next session. This significantly cut my Makie project runtime down when I tried it.

Apart from that, I also tried adding an image based approach where I bake in the dependencies to julia and store them all in cache in a single image. That way even a fresh session doesn’t need to precompile at all. Played around with it, and it seemed to work, but I dropped it again after realizing that at that point I am simply copying the precompilation cache and it would just end up with 200MB+ (Julia alone is 200mb) files on your harddrive that could also go stale… Anyway, something perhaps to come back to in the future. Maybe this could lead to another idea, like a julia image registry where we share base images like that…

19

^ this is imo a pretty good approach!

20

Thanks, and yeah, it seems to make a good difference.

But I really need some feedback from others on how it feels when they use it. Right now I am basically experimenting on my own, and even though I can mock scenarios, nothing beats working with it on real projects.