# Blog post: Julia's latency: Past, present and future
**URL:** https://discourse.julialang.org/t/blog-post-julias-latency-past-present-and-future/96910
**Category:** Community
**Created:** [March 31, 2023, 7:07pm UTC](https://discourse.julialang.org/t/blog-post-julias-latency-past-present-and-future/96910 "2023-03-31T19:07:45Z")
**Posts on this page:** 20
**Page:** 1
### Author:  [@jakobnissen](https://discourse.julialang.org/u/jakobnissen)
#### Post date: [March 31, 2023, 7:07pm UTC](https://discourse.julialang.org/t/blog-post-julias-latency-past-present-and-future/96910/1 "2023-03-31T19:07:45Z")
Dear all
I wrote a new blog post on Julia - this time on the history and future of Julia’s latency. Comments are welcome!
[https://viralinstruction.com/posts/latency](https://viralinstruction.com/posts/latency)
---
### Author:  [@mbauman](https://discourse.julialang.org/u/mbauman)
#### Post date: [March 31, 2023, 7:33pm UTC](https://discourse.julialang.org/t/blog-post-julias-latency-past-present-and-future/96910/2 "2023-03-31T19:33:38Z")
> Pre v1.0, focus was obviously on getting the semantics and API of the language right
One interesting thing that would be fun to reference here is that getting the semantics of the language right is one of the things that made latency worse. [Issue #265](https://github.com/JuliaLang/julia/issues/265) was perhaps the most infamous of Julia’s issue numbers, but its fix in v0.6 is precisely what introduced invalidations. But the connection between the #265 fix (a huge boon!) and latency (a huge boondoggle!) was lost on me — and I suspect many others — until Tim started on his latency hunts.
---
### Author:  [@aplavin](https://discourse.julialang.org/u/aplavin)
#### Post date: [March 31, 2023, 9:02pm UTC](https://discourse.julialang.org/t/blog-post-julias-latency-past-present-and-future/96910/3 "2023-03-31T21:02:02Z")
> ### [2: Avoid defining overly-broad methods](https://viralinstruction.com/posts/latency/#2_avoid_defining_overly-broad_methods)
section - doesn’t it go against the common and recommended approach of only putting type constraints in methods when (potentially) needed for dispatch?
---
### Author:  [@jakobnissen](https://discourse.julialang.org/u/jakobnissen)
#### Post date: [March 31, 2023, 9:09pm UTC](https://discourse.julialang.org/t/blog-post-julias-latency-past-present-and-future/96910/4 "2023-03-31T21:09:21Z")
Not adding type annotations is fine. The issue is when you add new methods that are _semantically_ too broad. In this example:
1. adding constructors to whole groups of types you don’t know anything about
2. Adding a convert method to types you don’t know the semantics of.
---
### Author:  [@jebej](https://discourse.julialang.org/u/jebej)
#### Post date: [March 31, 2023, 10:15pm UTC](https://discourse.julialang.org/t/blog-post-julias-latency-past-present-and-future/96910/5 "2023-03-31T22:15:07Z")
> Since Julia v1.0 (and before), Julia already cached compiled code in two places: Code compiled during a session was cached in memory, and code compiled during package installation was cached to disk in a process known as _precompilation_
I thought that precompilation was only caching parsed code (or parsed + one extra step) but not actually fully-compiled code.
EDIT: read the whole post and saw this was mentionned at the end…
---
### Author:  [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)
#### Post date: [March 31, 2023, 10:18pm UTC](https://discourse.julialang.org/t/blog-post-julias-latency-past-present-and-future/96910/6 "2023-03-31T22:18:03Z")
Prior to 1.9 it precompilation cached typed IR (after running type inference and Julia optimizations). In 1.9, it’s native code.
---
### Author:  [@jebej](https://discourse.julialang.org/u/jebej)
#### Post date: [March 31, 2023, 10:23pm UTC](https://discourse.julialang.org/t/blog-post-julias-latency-past-present-and-future/96910/7 "2023-03-31T22:23:38Z")
> [@Oscar\_Smith](#):
>
> In 1.9, it’s native code.
I just check the `compiled` folder and indeed, now on 1.9 there are `ji` and `dll` files. The `ji` files are much smaller than on 1.8, but in total the size increases by ~50% (for OrdinaryDiffEq at least).
It would be interesting to get insight into how this works & what is stored, maybe a blog post?
---
### Author:  [@mbauman](https://discourse.julialang.org/u/mbauman)
#### Post date: [March 31, 2023, 10:59pm UTC](https://discourse.julialang.org/t/blog-post-julias-latency-past-present-and-future/96910/8 "2023-03-31T22:59:42Z")
Jeff’s 1.9 webinar talks a bit about this.
> **[What's New in Julia 1.9 - JuliaHub](https://juliahub.com/company/resources/webinar/what-is-new-in-julia_1_9/)**
>
> What's New in Julia 1.9 - JuliaHub
---
### Author:  [@TheCedarPrince](https://discourse.julialang.org/u/TheCedarPrince)
#### Post date: [April 1, 2023, 2:09am UTC](https://discourse.julialang.org/t/blog-post-julias-latency-past-present-and-future/96910/9 "2023-04-01T02:09:04Z")
Oh man… New Jakob Nissen post just dropped. Thanks for sharing and eager to read!!! 😄
---
### Author:  [@tim.holy](https://discourse.julialang.org/u/tim.holy)
#### Post date: [April 1, 2023, 9:51am UTC](https://discourse.julialang.org/t/blog-post-julias-latency-past-present-and-future/96910/10 "2023-04-01T09:51:28Z")
Great post! You researched everything very carefully.
I have only one substantive correction: in [Julia's latency: Past, present and future](https://viralinstruction.com/posts/latency/#4_write_inferable_code), using Cthulhu’s `@descend` is now (IMO) significantly easier than `@code_warntype`; don’t forget that reading type-inferred code is basically reading a foreign language, and the new Cthulhu avoids the need by presenting results woven into the source code as written by the programmer.
Another small point is that the reasons you cite for our interpreter being slow are all true, but there’s another one: so much of Julia is written in Julia, it means there is much more to interpret. It’s very easy for Python to interpret `call_c_because_I_do_all_real_work_in_C(args...)`.
---
### Author:  [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)
#### Post date: [April 1, 2023, 11:49am UTC](https://discourse.julialang.org/t/blog-post-julias-latency-past-present-and-future/96910/11 "2023-04-01T11:49:18Z")
> [@tim.holy](#):
>
> Another small point is that the reasons you cite for our interpreter being slow are all true, but there’s another one: so much of Julia is written in Julia, it means there is much more to interpret. It’s very easy for Python to interpret `call_c_because_I_do_all_real_work_in_C(args...)`.
A mode where your code is interpreted but uses compiled package code would be very interesting.
---
### Author:  [@zgornel](https://discourse.julialang.org/u/zgornel)
#### Post date: [April 1, 2023, 11:51am UTC](https://discourse.julialang.org/t/blog-post-julias-latency-past-present-and-future/96910/12 "2023-04-01T11:51:11Z")
Great post, thanks! These posts look more and more like advanced complements to the language manual 🙂 and a good starting point for a book.
---
### Author:  [@DanielVandH](https://discourse.julialang.org/u/DanielVandH)
#### Post date: [April 1, 2023, 11:55am UTC](https://discourse.julialang.org/t/blog-post-julias-latency-past-present-and-future/96910/13 "2023-04-01T11:55:07Z")
Really well written! Having only a little understanding of latency and invalidations this was really interesting, and also taught me a lot. Well done
---
### Author:  [@tim.holy](https://discourse.julialang.org/u/tim.holy)
#### Post date: [April 1, 2023, 12:00pm UTC](https://discourse.julialang.org/t/blog-post-julias-latency-past-present-and-future/96910/14 "2023-04-01T12:00:57Z")
You can fake this with `push!(JuliaInterpreter.compiled_modules, Base, OtherModulesYouWantToRunCompiled...)`. Perhaps we should add an `JuliaInterpreter.compiled_modules_except(mods...)`.
But riffing off your point, we could choose interpret vs compile based on what is already compiled. 🤔
---
### Author:  [@Olivier\_Merchiers](https://discourse.julialang.org/u/Olivier_Merchiers)
#### Post date: [April 1, 2023, 12:18pm UTC](https://discourse.julialang.org/t/blog-post-julias-latency-past-present-and-future/96910/15 "2023-04-01T12:18:13Z")
Indeed the post is a great overview of the work done on latency. I Don’t know any of the technical details but it gives an idea of how challenging it was to reach those great achievements.
Sorry if I go slightly off topic but since
> [@tim.holy](#):
>
> using Cthulhu’s `@descend` is now (IMO) significantly easier than `@code_warntype`
Are there plans to replace the output of `@code_warntype ` in base by what `@descend` gives? That would be so great for beginners
---
### Author:  [@tim.holy](https://discourse.julialang.org/u/tim.holy)
#### Post date: [April 1, 2023, 12:24pm UTC](https://discourse.julialang.org/t/blog-post-julias-latency-past-present-and-future/96910/16 "2023-04-01T12:24:02Z")
Not in the near future, it’s a pretty huge dependency stack.
It will become much easier when the ponies are delivered to my doorstep: aka, when JuliaSyntax is Julia’s parser, and when lowering has been rewritten to track location info in the same way JuliaSyntax does for parsing, then nearly everything needed will already be in Julia proper. So at that point, I think that’s something we should just do.
But in the short term, we should stop recommending `@code_warntype` and just recommend Cthulhu.
---
### Author:  [@Olivier\_Merchiers](https://discourse.julialang.org/u/Olivier_Merchiers)
#### Post date: [April 1, 2023, 12:38pm UTC](https://discourse.julialang.org/t/blog-post-julias-latency-past-present-and-future/96910/17 "2023-04-01T12:38:03Z")
Thanks for the great explanation!
---
### Author:  [@jakobnissen](https://discourse.julialang.org/u/jakobnissen)
#### Post date: [April 1, 2023, 6:06pm UTC](https://discourse.julialang.org/t/blog-post-julias-latency-past-present-and-future/96910/18 "2023-04-01T18:06:29Z")
Thanks for the comments, I agree and have amended the post as suggested.
Also, the new Cthulhu is great as an advanced replacement of `@code_warntype`! Nice work.
---
### Author:  [@laborg](https://discourse.julialang.org/u/laborg)
#### Post date: [April 1, 2023, 6:40pm UTC](https://discourse.julialang.org/t/blog-post-julias-latency-past-present-and-future/96910/19 "2023-04-01T18:40:45Z")
I understand that the solution to issue #265 helps with interactive development in the REPL, but where else is this useful/necessary? The Julia community coped with non-redefinable types, so isn’t this similar to #265? Could we have a latency friendly Julia fork by sacrificing world-age?
---
### Author:  [@tim.holy](https://discourse.julialang.org/u/tim.holy)
#### Post date: [April 1, 2023, 7:32pm UTC](https://discourse.julialang.org/t/blog-post-julias-latency-past-present-and-future/96910/20 "2023-04-01T19:32:00Z")
The solution to #265 (aka, adding invalidation as a language feature) is needed if you want to support the following combination of properties:
1. interactive development
2. “method overloading” by packages that don’t own the function
3. aggressive compilation
4. consistent compilation: same result no matter how you got there
To illustrate: suppose you have a function `f` with one method,
```julia
f(::Any) = 1
```
and then write
```julia
g(list) = sum(f.(list))
```
Now let `list` be a `Vector{Any}`. You can compile a fast `g(::Vector{Any})` (_aggressive compilation_) by leveraging the fact that you know there’s only one possible method of `f`, and you know what the output is: `g(list)` gives you just `length(list)`. But now suppose you add a second method (_interactive development_ + _method overloading_)
```julia
f(::MyObj) = 2
```
where `MyObj` is some new type you’ve defined (so it’s not type-piracy). If you want to get the right answer (_consistent compilation_) from an arbitrary `list::Vector{Any}`, there are only two options:
a) plan for this eventuality from the beginning, by making every `f(::Any)` be called by runtime dispatch. But if there really is only one method of `f` this is vastly slower, so this at least partly violates _aggressive compilation_.
b) throw away the code for `g` that you created when there was only one method of `f`, and recompile it in this new world where there are two.
Julia does a mix of these: we do b) up to 3 methods, and then a) thereafter.
Now, I’ve framed this as an experiment at the REPL, but keep in mind this is also relevant if you load two packages: `PkgA` might define `f` and `g`, and `PkgB` might define a second method of `PkgA.f`. Unless you want to defer _all_ compilation (including for `Base`) until the entire session is loaded and then closed to further extension, you have to make the same choice between a) and b). The entire package ecosystem would collapse if we didn’t.
I’d argue that the combination of these four properties is a lot of what makes Julia what it is. AFAIK, Julia is the only language that supports all four of these properties. But that means we’re having to blaze new ground to figure out what the costs are and how to mitigate them.
[Next page](https://discourse.julialang.org/t/blog-post-julias-latency-past-present-and-future/96910.md?page=2)