# JIT Compiler for CPython

**URL:** <https://discourse.julialang.org/t/jit-compiler-for-cpython/108055>\
**Category:** Offtopic\
**Tags:** python\
**Created:** [December 26, 2023, 12:33pm UTC](https://discourse.julialang.org/t/jit-compiler-for-cpython/108055 "2023-12-26T12:33:04Z")\
**Posts on this page:** 14\
**Page:** 1

<div class="post-metadata">

**Author:** ![xiaoxi](https://avatars.discourse-cdn.com/v4/letter/x/a9adbd/32.png) [@xiaoxi](https://discourse.julialang.org/u/xiaoxi)\
**Post date:** [December 26, 2023, 12:33pm UTC](https://discourse.julialang.org/t/jit-compiler-for-cpython/108055/1 "2023-12-26T12:33:04Z")

</div>

Some Python core developers are floating the idea of a JIT compiler for Python.

[![](https://global.discourse-cdn.com/julialang/original/3X/3/7/375bf9bbc71ea85c7669783e044d51a442c45ca7.jpeg "Brandt Bucher – A JIT Compiler for CPython") ](https://www.youtube.com/watch?v=HxSHIpEQRjs)

By the way, the speaker has created a PR and used poetry to describe it.

> <https://github.com/python/cpython/pull/113465>
>
> 'Twas the night before Christmas, when all through the code
> Not a core dev was …merging, not even Guido;
> The CI was spun on the PRs with care
> In hopes that green check-markings soon would be there;
> The buildbots were nestled all snug under desks,
> Even PPC64 AIX;
> Doc-writers, triage team, the Council of Steering,
> Had just stashed every change and stopped engineering,
> 
> When in the "PRs" tab arose such a clatter,
> They opened GitHub to see what was the matter.
> Away to CPython they flew like a flash,
> Towards sounds of \`PROT\_EXEC\` and \`\_\_builtin\_\_\_clear\_cache\`.
> First \[LLVM was downloaded, unzipped\](https://github.com/brandtbucher/cpython/blob/justin/Tools/jit/README.md)
> Then the Actions were running \[a strange new build script\](https://github.com/brandtbucher/cpython/blob/justin/Tools/jit/build.py),
> When something appeared, they were stopped in their tracks,
> \[\`jit\_stencils.h\`\](https://gist.github.com/brandtbucher/d6397b1d106c55164b32df27d53eb7b8), generated from \[hacks\](https://github.com/brandtbucher/cpython/blob/justin/Tools/jit/template.c),
> With their spines all a-shiver, they muttered "Oh, shit...",
> They knew in a moment it must be a JIT.
> 
> More rapid than interpretation it came
> And it copied-and-patched every stencil by name:
> "Now, \`\_LOAD\_FAST\`! Now, \`\_STORE\_FAST\`! \`\_BINARY\_OP\_ADD\_INT\`!
> On, \`\_GUARD\_DORV\_VALUES\_INST\_ATTR\_FROM\_DICT\`!
> To the top of the loop! And down into the call!
> Now cache away! Cache away! Cache away all!"
> But why now? And how so? They needed a hint,
> Thankfully, Brandt gave a great talk at the sprint;
> So over to \[YouTube\](https://youtu.be/HxSHIpEQRjs) the reviewers flew,
> They read \[the white paper\](https://dl.acm.org/doi/10.1145/3485513), and \[the blog post\](https://sillycross.github.io/2023/05/12/2023-05-12) too.
> 
> And then, after watching, they saw its appeal
> Not writing the code themselves seemed so unreal.
> And the platform support was almost too easy,
> ARM64 Macs to 32-bit PCs.
> There was \[some runtime C\](https://github.com/brandtbucher/cpython/blob/justin/Python/jit.c), not too much, just enough,
> Basically a loader, relocating stuff;
> It ran every test, one by one passed them all,
> With not one runtime dependency to install.
> Mostly build-time Python! With strict static typing!
> For maintenance ease, and also nerd-sniping!
> 
> Though dispatch was faster, the JIT wasn't wise,
> And the traces it used still should be optimized;
> The code it was JIT'ing still needed some thinning,
> With code models small, and some register pinning;
> Or \[new calling conventions\](https://discourse.llvm.org/t/rfc-exposing-ghccc-calling-convention-as-preserve-none-to-clang/74233), shared stubs for paths slow,
> Since this JIT was brand new, there was fruit hanging low.
> It was awkwardly large, parsed straight out of the ELFs,
> And they laughed when they saw it, in spite of themselves;
> 
> A \`configure\` flag, and no merging this year,
> Soon gave them to know they had nothing to fear;
> It wasn't much faster, at least it could work,
> They knew that'd come later; no one was a jerk,
> But they were still smart, and determined, and skilled,
> They opened a shell, and configured the build;
> \`--enable-experimental-jit\`, then made it,
> And away the JIT flew as their "+1"s okay'ed it.
> But they heard it exclaim, as it traced out of sight,
> "Happy JIT-mas to all, and to all a good night!"
> 
> 
> \* Issue: gh-113464

---

<div class="post-metadata">

**Author:** ![xiaoxi](https://avatars.discourse-cdn.com/v4/letter/x/a9adbd/32.png) [@xiaoxi](https://discourse.julialang.org/u/xiaoxi)\
**Post date:** [January 9, 2024, 3:18pm UTC](https://discourse.julialang.org/t/jit-compiler-for-cpython/108055/2 "2024-01-09T15:18:41Z")

</div>

For those who prefer to read rather than watch a video, this article explains how the Python JIT compiler works.

> **[Python 3.13 gets a JIT](https://tonybaloney.github.io/posts/python-gets-a-jit.html)**
>
> Reviewing the JIT in Python 3.13

Out of curiosity, what are the pros and cons of the Julia JIT compiler versus the Python copy-and-patch JIT compiler?

---

<div class="post-metadata">

**Author:** ![sylvaticus](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sylvaticus/32/203883_2.png) [@sylvaticus](https://discourse.julialang.org/u/sylvaticus)\
**Post date:** [January 9, 2024, 3:39pm UTC](https://discourse.julialang.org/t/jit-compiler-for-cpython/108055/3 "2024-01-09T15:39:03Z")

</div>

> [@xiaoxi](#):
>
> Out of curiosity, what are the pros and cons of the Julia JIT compiler versus the Python copy-and-patch JIT compiler?

A short, non technical, answer could be that Julia has been designed after LLVM went out 🙂

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [January 9, 2024, 4:10pm UTC](https://discourse.julialang.org/t/jit-compiler-for-cpython/108055/4 "2024-01-09T16:10:42Z")

</div>

> [@xiaoxi](#):
>
> Out of curiosity, what are the pros and cons of the Julia JIT compiler versus the Python copy-and-patch JIT compiler?

From the article:

> The initial benchmarks show something of [a 2-9% performance improvement](https://github.com/python/cpython/pull/113465#issuecomment-1876225775).

The basic issue is that it’s hard to efficiently compile Python code because the language semantics weren’t designed for this. There have been many attempts to JIT-compile Python, and some of them have been very sophisticated with impressive results (PyPy, Numba, Pythran, Pyston, …), but generally they have worked well only for a subset of the language. But Python is so widely used that even small speedups (or big speedups on a small subset) are worth the effort, and I wish them well.

It’s for the same reason that you can’t just slap a “Julia backend” underneath Python and expect to see any improvements — a compiler by itself isn’t enough, and Julia’s compiler (which is just ordinary LLVM) isn’t what makes Julia special. This is also a [Julia FAQ](https://docs.julialang.org/en/v1/manual/faq/#Why-don't-you-compile-Matlab/Python/R/%E2%80%A6-code-to-Julia?).

See also many previous discussions — [How hard would it be to implement Numpy.jl, i.e. Numpy in Julia?](https://discourse.julialang.org/t/how-hard-would-it-be-to-implement-numpy-jl-i-e-numpy-in-julia/22080) — [Python to Julia transpiler](https://discourse.julialang.org/t/python-to-julia-transpiler/21473) — [Convert Matlab Code to Julia 1.0](https://discourse.julialang.org/t/convert-matlab-code-to-julia-1-0/21529) — as well as [this blog post](http://www.stochasticlifestyle.com/why-numba-and-cython-are-not-substitutes-for-julia/) by @ChrisRackauckas.

---

<div class="post-metadata">

**Author:** ![xiaoxi](https://avatars.discourse-cdn.com/v4/letter/x/a9adbd/32.png) [@xiaoxi](https://discourse.julialang.org/u/xiaoxi)\
**Post date:** [January 9, 2024, 4:54pm UTC](https://discourse.julialang.org/t/jit-compiler-for-cpython/108055/5 "2024-01-09T16:54:07Z")

</div>

It appears that Python will have low startup time using the ideas of this paper.  
 ![image](https://global.discourse-cdn.com/julialang/original/3X/f/1/f1466b23cee302e185c4396cf51e8d8fdd2add4a.png)

> **[Copy-and-patch compilation: a fast compilation algorithm for high-level...](https://dl.acm.org/doi/10.1145/3485513)**
>
> Fast compilation is important when compilation occurs at runtime, such as query compilers
> in modern database systems and WebAssembly virtual machines in modern browsers. We
> present copy-and-patch, an extremely fast compilation technique that also ...

I just wonder why the Julia compiler doesn’t use this kind of JIT compiler.

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [January 10, 2024, 3:25am UTC](https://discourse.julialang.org/t/jit-compiler-for-cpython/108055/6 "2024-01-10T03:25:22Z")

</div>

Do you happen to have a few million dollars lying around to spend on changing JIT compilers?

---

<div class="post-metadata">

**Author:** ![Benny](https://avatars.discourse-cdn.com/v4/letter/b/49beb7/32.png) [@Benny](https://discourse.julialang.org/u/Benny)\
**Post date:** [January 10, 2024, 5:22am UTC](https://discourse.julialang.org/t/jit-compiler-for-cpython/108055/7 "2024-01-10T05:22:42Z")

</div>

> [@xiaoxi](#):
>
> It appears that Python will have low startup time using the ideas of this paper.

The paper isn’t about Python at all, but WebAssembly and a C++ -based DSL, the latter being the subject of that plot. Different language semantics would almost certainly affect the startup time, and the comparison of their own compiler to LLVM over their own DSL raises an unanswered question of whether their DSL introduces difficulties to compilation and optimization by LLVM. Since LLVM still makes faster code than their copy-and-patch compiler in their selected languages and Python’s copy-and-patch compiler so far made a negligible change to performance, there’s not a lot of justification for Julia to follow suit.

---

<div class="post-metadata">

**Author:** ![Wispy](https://avatars.discourse-cdn.com/v4/letter/w/a587f6/32.png) [@Wispy](https://discourse.julialang.org/u/Wispy)\
**Post date:** [January 11, 2024, 1:05pm UTC](https://discourse.julialang.org/t/jit-compiler-for-cpython/108055/8 "2024-01-11T13:05:01Z")

</div>

What would be the advantage of using something like this in Julia given that optimized Julia is already comparable to C? Would it help for unoptimized code?

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [January 11, 2024, 1:25pm UTC](https://discourse.julialang.org/t/jit-compiler-for-cpython/108055/9 "2024-01-11T13:25:45Z")

</div>

> [@Wispy](#):
>
> What would be the advantage of using something like this in Julia given that optimized Julia is already comparable to C? Would it help for unoptimized code?

As I understand it, the claim is not that they generate faster code, but that they generate code faster — that is, the compilation time is reduced, not the runtime of the resulting code. In fact, their resulting runtime is slightly worse, but they claim orders of magnitude improvement in compile time.

They do this by caching mostly compiled code for thousands of small snippets corresponding to fragments of ASTs (abstract syntax trees) that are frequently re-used:

> At a high level, copy-and-patch works by having a pre-built library of composable and parametrizable binary code snippets that we call binary stencils. At runtime, optimization and code generation become the simple task of looking up a data table to select the appropriate stencil, and instantiate it to the desired position by copying it and patching in the missing values. […] The stencil library contains many stencil variants for each bytecode or AST node type that are specialized for different operand types, value locations, and more.

It’s not exactly clear to me what the granularity of their stencils are — they cache about 100,000 of them, but only give a handful of examples, like:  
 ![image](https://global.discourse-cdn.com/julialang/original/3X/5/1/51fb5830bd6ed4adfee3fe2c98017deee6ff4f51.png)

Julia’s code generator also caches code for thousands of small snippets, at the granularity of function calls specialized for different argument types. e.g. `a == b` is a function call in Julia with many different compiled variants, and the cached code is often inlined at the call site. However, the form in which Julia caches the code (typed SSA-form ASTs?) is much higher level than what the copy-and-patch algorithm uses, I think.

---

<div class="post-metadata">

**Author:** ![klwlevy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/klwlevy/32/25272_2.png) [@klwlevy](https://discourse.julialang.org/u/klwlevy)\
**Post date:** [April 16, 2024, 9:45am UTC](https://discourse.julialang.org/t/jit-compiler-for-cpython/108055/10 "2024-04-16T09:45:32Z")

</div>

Now there is a PEP and some discussions about this here: [PEP 744 – JIT Compilation | peps.python.org](https://peps.python.org/pep-0744/)

> **[PEP 744: JIT Compilation](https://discuss.python.org/t/pep-744-jit-compilation/50756)**
>
> PEP 744 is an informational PEP answering many common questions about CPython 3.13’s new experimental JIT compiler. My main goal for this PEP is to build community consensus around the specific criteria that the JIT should meet in order to become...

---

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [April 16, 2024, 5:42pm UTC](https://discourse.julialang.org/t/jit-compiler-for-cpython/108055/11 "2024-04-16T17:42:04Z")

</div>

> [@Wispy](#):
>
> What would be the advantage of using something like this in Julia given that optimized Julia is already comparable to C?

Welcome Wispy to Julia, the advantage would smaller compiled Julia programs, and, indirectly in some cases, faster Julia programs.

Note, Julia is already the fastest dynamic language, after static (classic) Fortran:

![image](https://global.discourse-cdn.com/julialang/original/3X/1/f/1f4cd7e4fdb26d25f92df35fc7e72b88986697b3.png)

Until recently (i.e. with previous Julia version), Julia was compared to Java there at the Benchmark Game, but now to Fortran (and Chapel and C++), the current next fastest language(s).

We can fix that graph by working on the outlier(s), the top one (I think only one or two to beat Fortran); and the fixed startup-cost highlighed by the lowest outlier.

Julia isn’t really slower than C, Rust or C++. That’s an illusion. They compile ahead of time (e.g. with the slow LLVM), but the rules there only allow source code for Julia and thus Julia has the cost of compiling (JIT, and Julia’s JIT is slower than it needs to be) on the fly added to its runtime.

> **[Julia vs Classic Fortran - Which programs are fastest?](https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/julia.html)**
>
> Julia Classic Fortran - Which programs have fastest performance?

E.g. this one can be improved:

> **[fannkuch-redux - Which programs are fastest? (Benchmarks Game)](https://benchmarksgame-team.pages.debian.net/benchmarksgame/performance/fannkuchredux.html)**
>
> fannkuch-redux - Which programs are fastest? — How fast programs repeatedly access a tiny integer-sequence.

> [@Is it time to make LLVM optional and how?](https://discourse.julialang.org/t/is-it-time-to-make-llvm-optional-and-how/112677):
>
> LLVM is a very heavy dependency, and is required (for fully compiled apps), just (I meant e.g.) in case eval is used. I would like to drop LLVM requirement (for still rather fast code), and I know it’s possible, then you just (currently) risk a runtime error. Or you can use “min” here (but it’s very slow at runtime since all your code in interpreted): --compile={yes\*|no|all|min} Enable or disable JIT compiler, or request exhaustive or minimal compilation When I say …

---

<div class="post-metadata">

**Author:** ![tp2750](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tp2750/32/207806_2.png) [@tp2750](https://discourse.julialang.org/u/tp2750)\
**Post date:** [June 22, 2024, 8:07am UTC](https://discourse.julialang.org/t/jit-compiler-for-cpython/108055/12 "2024-06-22T08:07:08Z")

</div>

A recent report on the project: [Adding a JIT compiler to CPython [LWN.net]](https://lwn.net/SubscriberLink/977855/804f87c6cf5e5f56/).

The linked blog-post is very interesting: [Building a baseline JIT for Lua automatically |](https://sillycross.github.io/2023/05/12/2023-05-12/)

---

<div class="post-metadata">

**Author:** ![tp2750](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tp2750/32/207806_2.png) [@tp2750](https://discourse.julialang.org/u/tp2750)\
**Post date:** [June 22, 2024, 8:07am UTC](https://discourse.julialang.org/t/jit-compiler-for-cpython/108055/13 "2024-06-22T08:07:51Z")

</div>

> [@StefanKarpinski](#):
>
> Do you happen to have a few million dollars lying around to spend on changing JIT compilers?

I wish I had… It must be frustrating to see all these resources poured into python, which is not made for this kind of work, while julia clearly is.

---

<div class="post-metadata">

**Author:** ![fdekerme](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fdekerme/32/43574_2.png) [@fdekerme](https://discourse.julialang.org/u/fdekerme)\
**Post date:** [July 24, 2025, 1:10am UTC](https://discourse.julialang.org/t/jit-compiler-for-cpython/108055/14 "2025-07-24T01:10:03Z")

</div>

For those interested, a nice article summarizing Bucher’s last talk about the current state of Python JIT:

> **[Following up on the Python JIT](https://lwn.net/Articles/1029307/)**
>
> Performance of Python programs has been a major focus of development for the language over the \[...\]