# JIT issues relevant to realtime applications

**URL:** <https://discourse.julialang.org/t/jit-issues-relevant-to-realtime-applications/67736>\
**Category:** New to Julia\
**Tags:** performance, jit, c\
**Created:** [September 6, 2021, 7:30am UTC](https://discourse.julialang.org/t/jit-issues-relevant-to-realtime-applications/67736 "2021-09-06T07:30:25Z")\
**Posts on this page:** 20\
**Page:** 2

<div class="post-metadata">

**Author:** ![anon56330260](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@anon56330260](https://discourse.julialang.org/u/anon56330260)\
**Post date:** [September 7, 2021, 12:58pm UTC](https://discourse.julialang.org/t/jit-issues-relevant-to-realtime-applications/67736/21 "2021-09-07T12:58:08Z")

</div>

> [@Palli](#):
>
> you can precompile Julia code (and Julia without runtime code generation, i.e. without LLVM, is coming).

So Julia will have something other static languages already have for a long time, this is really not an advantage…

> [@Palli](#):
>
> and be better, and likely easier to develop from scratch that it was in Java.

I don’t know where you get this impression from. For the numerical part this may be true (and that’s one of the reason why Java Minecraft is so slow). But other parts like network, GUI will be another story.

> [@Palli](#):
>
> but neither a killer disadvantage over C# (usually used with a C++ game engine,

This is incorrect. I think a large part of Unity is written in C#, not C++ and you definitely can’t just use ir from Julia. Microsoft has specialized compiler for C# and you can directly write and compile shader in C#. This is definitely a killer advantage. And static checking is another one. Besides, C# also have some features for high performance code while Julia doesn’t have. For example, they support `stackalloc` keyword to force stack allocation.

Still, it’s possible to code a game engine in Julia and many 3A companies use their private game engine instead of the open source one (Unity or Unreal). Rust also has a lot of game engine, 3D, 2D or even textual one. But I don’t think Julia will become a good choice for game developing. It lacks too many infrastrutures and it offers no special benefit than other languages.

> [@Palli](#):
>
> still people prefer Julia for hard-real-time robotics to Java or C++. See for humanoid robots:

I think some MIT researchers choose Julia, the mainstream adopters are still C++.

---

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [September 7, 2021, 4:41pm UTC](https://discourse.julialang.org/t/jit-issues-relevant-to-realtime-applications/67736/22 "2021-09-07T16:41:06Z")

</div>

My point with “people prefer Julia for hard-real-time” is that it’s possible. Mindshare may be missing. Games, which as (soft) real-time, are also possible, I didn’t say easier than with C#, I meant basically no huge road-block.

You can precompile Julia code already, and not use runtime code generation. Getting rid of the LLVM as a dependency is only about distributing smaller binaries. Julia as is isn’t that large compared to huge games…

> [@anon56330260](#):
>
> This is incorrect. […] For example, they support `stackalloc` keyword to force stack allocation.

Interesting, while possibly not always. Julia stack allocates by default, since Julia 1.5, for isbits types. It’s likely `stackalloc` in C# is implemented by alloca (non-portable in C), \_alloca, or \_malloca, then possibly not stack-allocating. [Dos and Don'ts of stackalloc](https://vcsjones.dev/stackalloc/)

For [\_malloca | Microsoft Learn](https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/malloca?view=msvc-160)

> If _size_ is greater than **\_ALLOCA\_S\_THRESHOLD** , then **\_malloca** attempts to allocate on the heap, and returns a null pointer if the space can’t be allocated.

For C: [C Language Tutorial =\> alloca: allocate memory on stack](https://riptutorial.com/c/example/28419/alloca--allocate-memory-on-stack)

> Do not use `alloca()` in new code

In Linux: “The **alloca** () function is machine- and compiler-dependent.”, additionally in FreeBSD: “its use is discouraged.” Rust has at least two libraries: [stackalloc — Rust library // Lib.rs](https://lib.rs/crates/stackalloc)

> [@anon56330260](#):
>
> But other parts like network, GUI will be another story.

Games make their own “GUI”. I’m not aware of missing support do get graphics to the screen (at least for something comparable to Minecraft), and even if something is missing you can reuse C++ code to do that. I agree access to major game engines would be a plus, but you can do 3D only with Julia.

> **[GitHub - JuliaMultimedia/CSFML.jl: Julia bindings for CSFML](https://github.com/JuliaMultimedia/CSFML.jl)**
>
> Julia bindings for CSFML. Contribute to JuliaMultimedia/CSFML.jl development by creating an account on GitHub.

> It provides access to windowing, graphics, audio and network.

I’m not saying writing shaders is better in Julia (than e.g. C#). I doubt it, but it’s possible (or use C++ with):

> [@Writing GPU Shaders in Julia](https://discourse.julialang.org/t/writing-gpu-shaders-in-julia/60669):
>
> I was going over the [GLAbstraction tutorials](https://github.com/JuliaGL/GLAbstraction.jl/tree/master/tutorials) to learn how do do simple animations on the GPU, and things work fairly well. One pain point, however, is that the shaders needs to be written in a special language, meaning that they clash with the rest of the code. Is it possible to compile shaders from julia code? Rust has [rust-gpu](https://github.com/EmbarkStudios/rust-gpu), but I could not find anything similar for Julia.

---

<div class="post-metadata">

**Author:** ![solaris](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/solaris/32/28875_2.png) [@solaris](https://discourse.julialang.org/u/solaris)\
**Post date:** [September 7, 2021, 8:01pm UTC](https://discourse.julialang.org/t/jit-issues-relevant-to-realtime-applications/67736/23 "2021-09-07T20:01:58Z")

</div>

I’ll respond as to the interest in my part in Julia for game scripting. I’ve been in the game industry my entire career spanning a couple of decades, mostly triple A titles, including some very big games everyone knows.  
The core engine I’m using is written in C++ so Julia is being looked at as a scripting language solution for scripting designers primarily. Scripting designers need to know some coding but not as much or in depth as a regular programmer; similar to what’s expected from a Unity developer as far as C# is concerned.

For this audience, there’s a balance to reach between the scripting language’s ability to express their designs, and simplicity; really the language’s ability to protect designers from themselves. Something dynamic where a designer doesn’t have to restart the game after making a change to their scripts is ideal. This alone excludes a lot of scripting languages.

It would also be great if the language was not statically typed and if memory management was automatic (which requires GC). This also excludes a lot of languages. The GC should be as fast and lightweight as possible. The full-on generational GC that copies tons of data and creates unpredictable stalls, such as C#, Java or Python would be undesirable. As an aside, I spent a couple of years charging major publishers an arm and a leg to fix their GC issues in Unity games.

There are other options for game scripting such as Lua, but the performance tradeoff can be immense. In my testing environment Lua has 33-200x worse performance than C++. Ideally, designers need to be able to write code and not have to call someone like me to come and turn it into C++ at some point for performance reasons.

Hashlink is decent, a bit less beginner friendly than other options, a bit less performant than others. Angelscript/Squirrel are a bit outdated and not really supported anymore.

All in all, if the PackageCompiler solves the loading issue, I’d rather use Julia because it’s super fast, dynamic, decent GC, and semi-loosely typed – unless I’m missing something here.

---

<div class="post-metadata">

**Author:** ![jpsamaroo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jpsamaroo/32/46804_2.png) [@jpsamaroo](https://discourse.julialang.org/u/jpsamaroo)\
**Post date:** [September 7, 2021, 8:44pm UTC](https://discourse.julialang.org/t/jit-issues-relevant-to-realtime-applications/67736/24 "2021-09-07T20:44:38Z")

</div>

I think Julia is basically what you want, but we have some work to do to make the GC and JIT behavior more latency-friendly, as well as changes to the compiler to make AOT compilation easier and faster:

GC improvements:

[https://github.com/JuliaLang/julia/pull/41760](https://github.com/JuliaLang/julia/pull/41760) should reduce GC latency on multithreaded systems by making all threads cooperate during marking (the most expensive GC step).

Work on escape analysis in the compiler should eventually allow object finalizers to be run immediately at the end of a scope containing a non-escaping object, which should reduce issues with memory consumption for heap-allocated objects with attached finalizers.

[https://github.com/JuliaLang/julia/pull/41616](https://github.com/JuliaLang/julia/pull/41616) will emit more GC safepoints, reducing stop-the-world latency for multithreaded programs, although further work is needed to improve this situation.

More GC generations could be added to reduce how many non-incremental GC passes need to be performed (as I understand it).

JIT improvements:

Multithreaded compilation should make it possible to minimize JIT latency on UI threads. I suspect this will be a very difficult PR, but will be endlessly beneficial.

Compiler improvements:

[https://github.com/JuliaLang/julia/pull/41936](https://github.com/JuliaLang/julia/pull/41936) will allow the compiler and LLVM to be removed from PackageCompiler binaries which don’t require the JIT at runtime.

[https://github.com/JuliaLang/julia/pull/40414](https://github.com/JuliaLang/julia/pull/40414) will make sysimage generation drastically faster, making it easier to iterate on PackageCompiler-generated binaries.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [September 8, 2021, 7:47am UTC](https://discourse.julialang.org/t/jit-issues-relevant-to-realtime-applications/67736/25 "2021-09-08T07:47:44Z")

</div>

> [@solaris](#):
>
> a scripting language solution for designers primarily

I would go with a lightweight Scheme/Lisp that has good C (C++) bindings.

Julia is not really a good fit for a scripting language; people use it of course for scripting because they are familiar with it, but if you are yet to make the investment into a new language then perhaps Julia is overkill for this purpose.

---

<div class="post-metadata">

**Author:** ![solaris](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/solaris/32/28875_2.png) [@solaris](https://discourse.julialang.org/u/solaris)\
**Post date:** [September 8, 2021, 8:47am UTC](https://discourse.julialang.org/t/jit-issues-relevant-to-realtime-applications/67736/26 "2021-09-08T08:47:17Z")

</div>

There’s a couple of problems with something like Lisp or Scheme. First, designers are overwhelmingly trained in traditional C-style type of languages. Lisp-style languages would seem very alien. Second is performance. If your team of designers need to hire a high pay engineer to turn their code into c++ which becomes a black box that few dare mess with, then the scripting language has failed at its job.  
For a few years I did just that. Going from developer to developer charging an arm and a leg from people using languages that didn’t scale and/perform well. A lot of Unity and Unreal projects.  
I went into a big project with Sony that was using Lua and they told me, “the game runs at 8 fps. We need it to run at 60”. It mainly involved ripping Lua out.  
Speed, dynamic (doesn’t need to be compiled), and the ability to express their designs is difficult to find.  
Being lightweight is actually not that important (i mean, up to a point…i wouldn’t dare try JS). C# is one of the most popular game scripting languages because of Unity and I’d say Julia is more lightweight than that.  
My second choice is Cython, but to get good performance it has to be converted to C and even though that process can be automated, it would lose its dynamic nature, and debugging it would be terrible.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [September 8, 2021, 9:05am UTC](https://discourse.julialang.org/t/jit-issues-relevant-to-realtime-applications/67736/27 "2021-09-08T09:05:15Z")

</div>

> [@solaris](#):
>
> Lisp-style languages would seem very alien. Second is performance.

Possibly, but surface syntax is just one aspect of a language. Julia is much closer to Common Lisp underneath (multiple dispatch), and in practice requires quite a bit of learning not only to reap its performance benefits, but to avoid writing code that taxes the compiler for no good benefit.

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [September 8, 2021, 1:10pm UTC](https://discourse.julialang.org/t/jit-issues-relevant-to-realtime-applications/67736/28 "2021-09-08T13:10:44Z")

</div>

I’ve never seen a thread with so much people trying to dissuade someone to use Julia here 😂 😂

But since we are there:

> [@solaris](#):
>
> doesn’t have to restart the game after making a change to their scripts is ideal. This alone excludes a lot of scripting languages.

Note that Julia has to be restarted if you change constant values (`const` variables) and if you change the structure of a `struct` (number and/or types of the fields).

Anyway, you seem to know the field you are willing to test it very well. You can probably try it out, and report your experience, good or bad, we will all learn something with that.

---

<div class="post-metadata">

**Author:** ![dlakelan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlakelan/32/8491_2.png) [@dlakelan](https://discourse.julialang.org/u/dlakelan)\
**Post date:** [September 8, 2021, 3:51pm UTC](https://discourse.julialang.org/t/jit-issues-relevant-to-realtime-applications/67736/29 "2021-09-08T15:51:28Z")

</div>

Well if we’re going there, I’m going to chip in with my thought that Julia in a game is certainly better than a lot of other dynamic languages would be. I think it’s absolutely worth trying out!

---

<div class="post-metadata">

**Author:** ![solaris](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/solaris/32/28875_2.png) [@solaris](https://discourse.julialang.org/u/solaris)\
**Post date:** [September 8, 2021, 5:32pm UTC](https://discourse.julialang.org/t/jit-issues-relevant-to-realtime-applications/67736/30 "2021-09-08T17:32:31Z")

</div>

This is actually great dialog, mostly constructive 👍

So if I change a struct in a module, in an embedded scenario, do I have to restart the whole Julia session or can I just reload the module the struct is in?

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [September 8, 2021, 5:48pm UTC](https://discourse.julialang.org/t/jit-issues-relevant-to-realtime-applications/67736/31 "2021-09-08T17:48:41Z")

</div>

In pure julia, yes (even with Revise.jl). This isn’t tied to being embedded or not (in fact, aside from starting the REPL, “pure” julia doesn’t differ from embedded julia at all as far as I know).

Pluto.jl does some shenanigans to make that replacement kind-of work, but I think this comes at the cost of recompiling existing functions (which you kinda have to do, since the struct layout may have changed arbitrarily and thus all old compiled code relying on old field positions in memory etc would break).

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [September 8, 2021, 5:55pm UTC](https://discourse.julialang.org/t/jit-issues-relevant-to-realtime-applications/67736/32 "2021-09-08T17:55:41Z")

</div>

> [@solaris](#):
>
> So if I change a struct in a module, in an embedded scenario, do I have to restart the whole Julia session or can I just reload the module the struct is in?

Yes. There are some preliminary developing-workflow workarounds, like using named tuples (like this: [GitHub - BeastyBlacksmith/ProtoStructs.jl: Easy prototyping of structs](https://github.com/BeastyBlacksmith/ProtoStructs.jl)). But in general when one is changing very frequently the custom types, one need to do frequent restarts.

---

<div class="post-metadata">

**Author:** ![JTriggerFish](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jtriggerfish/32/51516_2.png) [@JTriggerFish](https://discourse.julialang.org/u/JTriggerFish)\
**Post date:** [July 24, 2023, 4:44pm UTC](https://discourse.julialang.org/t/jit-issues-relevant-to-realtime-applications/67736/33 "2023-07-24T16:44:12Z")

</div>

Revisiting this topic in 2023, is there any way to stop the JIT from interfering with every single thread ?  
I’m testing some interractive audio generation where I have an audio thread running and a main thread for user interraction and as far as I can tell it seems that JIT triggered by the main thread interrupts the audio thread even when it compiles things that don’t directly affect it.

---

<div class="post-metadata">

**Author:** ![ufechner7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ufechner7/32/51363_2.png) [@ufechner7](https://discourse.julialang.org/u/ufechner7)\
**Post date:** [July 24, 2023, 4:57pm UTC](https://discourse.julialang.org/t/jit-issues-relevant-to-realtime-applications/67736/34 "2023-07-24T16:57:27Z")

</div>

Just make sure that JIT does not happen… You only need a (pre-compile) script that calls all functions with all possible type combinations…

---

<div class="post-metadata">

**Author:** ![JTriggerFish](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jtriggerfish/32/51516_2.png) [@JTriggerFish](https://discourse.julialang.org/u/JTriggerFish)\
**Post date:** [July 24, 2023, 5:10pm UTC](https://discourse.julialang.org/t/jit-issues-relevant-to-realtime-applications/67736/35 "2023-07-24T17:10:29Z")

</div>

I don’t know if that was tongue in cheek given it doesn’t seem trivial to do 🙂  
I tried for even a very simple case and it’s proving difficult in practice. For instance I use eval() and want to catch errors and display stack traces, and I wrote a warmup function that purposedly throws before the audio is started.  
That works if I write to stdout, but I don’t want to do that so I used an IOBuffer that does nothing, but then the first actual error still introduces a big pause.

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [July 24, 2023, 5:45pm UTC](https://discourse.julialang.org/t/jit-issues-relevant-to-realtime-applications/67736/36 "2023-07-24T17:45:10Z")

</div>

> [@JTriggerFish](#):
>
> For instance I use eval() and want to catch errors and display stack traces

`eval` by definition needs compiler, so you can’t statically compile it.

---

<div class="post-metadata">

**Author:** ![JTriggerFish](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jtriggerfish/32/51516_2.png) [@JTriggerFish](https://discourse.julialang.org/u/JTriggerFish)\
**Post date:** [July 24, 2023, 5:48pm UTC](https://discourse.julialang.org/t/jit-issues-relevant-to-realtime-applications/67736/37 "2023-07-24T17:48:14Z")

</div>

Of course, I’m not tring to statically compile it, I’m trying to _at runtime_ call it before the audio thread is started, with a few different arguments, to warm up the JIT for stuff where it particularly matters.  
My point was that even that doesn’t really work or at least is very hard to do properly.  
But coming back to my initial question, I guess it’s not currently possible ?

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [July 24, 2023, 5:50pm UTC](https://discourse.julialang.org/t/jit-issues-relevant-to-realtime-applications/67736/38 "2023-07-24T17:50:08Z")

</div>

basically the generally pattern is your have a `.jl` script that takes some input, you compile it with dummy input, and you get a binary you can run later without Julia compiler.

There’s no room to have a `eval()` anywhere – you just can’t have eval involved in this process

---

<div class="post-metadata">

**Author:** ![JTriggerFish](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jtriggerfish/32/51516_2.png) [@JTriggerFish](https://discourse.julialang.org/u/JTriggerFish)\
**Post date:** [July 24, 2023, 6:45pm UTC](https://discourse.julialang.org/t/jit-issues-relevant-to-realtime-applications/67736/39 "2023-07-24T18:45:33Z")

</div>

Yes I understand, but that wasn’t quite what I was asking. I guess the answer to the thread interruption is ‘no’.

---

<div class="post-metadata">

**Author:** ![jpsamaroo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jpsamaroo/32/46804_2.png) [@jpsamaroo](https://discourse.julialang.org/u/jpsamaroo)\
**Post date:** [July 24, 2023, 7:15pm UTC](https://discourse.julialang.org/t/jit-issues-relevant-to-realtime-applications/67736/40 "2023-07-24T19:15:42Z")

</div>

Responders are being confused here; @JTriggerFish _cannot_ just precompile everything ahead of time, the nature of what they’re doing requires runtime JIT capabilities (for interactive audio synthesis/processing, which sounds very cool!).

Julia’s JIT is not yet “interactive-friendly”, and even once it is, our GC is not. I would consider what was suggested (by you?) in another Discourse post, which is to put code generation into another Julia process. You will then need to figure out how to “ship” that code into your real-time process, potentially by using StaticCompiler.jl to generate a linkable binary? Or you could compile to virtual ISA like WASM and have your real-time process run a WASM VM.

[Previous page](https://discourse.julialang.org/t/jit-issues-relevant-to-realtime-applications/67736.md?page=1)

[Next page](https://discourse.julialang.org/t/jit-issues-relevant-to-realtime-applications/67736.md?page=3)