# Roadmap for a faster time-to-first-plot?

**URL:** <https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956>\
**Category:** Internals & Design\
**Tags:** ttfp\
**Created:** [April 9, 2019, 1:22pm UTC](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956 "2019-04-09T13:22:42Z")\
**Posts on this page:** 20\
**Page:** 9

<div class="post-metadata">

**Author:** ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)\
**Post date:** [August 25, 2019, 12:03am UTC](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956/161 "2019-08-25T00:03:03Z")

</div>

Packagecompiler is very often mentioned as a solution to these kind of problems , but it really is not.

1. The issue page is very long [https://github.com/JuliaLang/PackageCompiler.jl/issues](https://github.com/JuliaLang/PackageCompiler.jl/issues), full of issues where it is not working.
2. the author of PackageCompiler.jl made it very clear in his juliacon talk that he is not interested in maintaining the repository (understandably)

Altogether, I don’t think this can be considered a solution to anything. It can somewhat mitigate your problems if you are lucky and willing to spend some effort solving the problems that arise from its application, but the TTFP-problem would still benefit greatly from the planned compiler latency work. Effort going into the latency improvements will also have an immediate impact on all code for all users 😍

For now, the best mitigation I’ve found is going through lengths to make sure you do not have to restart Julia, together with the juno cycler to keep an instance ready when Julia does crash

---

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [August 25, 2019, 12:45am UTC](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956/162 "2019-08-25T00:45:16Z")

</div>

I find myself hoping that a package compiler style solution is found instead of putting all the focus on just reducing compile times in general. My (possibly unfounded) concern is that if we as a community constantly complain about compile times, it might lead the devs to make runtime performance sacrifices in the name of compiler latency.

Personally, I’d rather the opposite and would gladly take significantly longer compile times in exchange for a tiny boost in runtime speed. If the compiler devs can make _both_ faster, then great. However, I know that if the two come into conflict I’d take runtime improvements any day of the week. Given that, I’d much rather an approach that just lets me re-use or even share compile work so that I run into the compiler less often and then it wouldn’t be so painful to crank up compile times.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [August 25, 2019, 1:00am UTC](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956/163 "2019-08-25T01:00:22Z")

</div>

> [@Mason](#):
>
> Personally, I’d rather the opposite and would gladly take significantly longer compile times in exchange for a tiny boost in runtime speed. If the compiler devs can make _both_ faster, then great. However, I know that if the two come into conflict I’d take runtime improvements any day of the week. Given that, I’d much rather an approach that just lets me re-use or even share compile work so that I run into the compiler less often and then it wouldn’t be so painful to crank up compile times.

The analysis above already describes exactly what the tradeoff is. Type-stable code is fine for run time and compile time speed. When you have things start to not be type stable, the compiler seems to have to work a lot harder to find out what is the smallest union you could want, and whether that could be used to get any little optimizations. If the compiler just gives up and goes to `Any`, in many cases there could be no run time cost, but it would definitely make the compiler faster. But some users might think there’s a regression since some codes that partially inferred would just shoot to `Any`. IMO that is the right behavior, but I can see the downsides.

In my view, I would like macros that act like pragmas to control how much the compiler should try, defaulting towards faster compile in this case.

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [August 25, 2019, 2:38am UTC](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956/164 "2019-08-25T02:38:10Z")

</div>

> [@ChrisRackauckas](#):
>
> - If you can, use things in the system image. `Dict{Symbol,MyType{T}} where T` will require compiling all of the little parts, while `Dict{Any,Any}` is used in Julia so it won’t.

Will it be solved when/if this PR is landed?

> <https://github.com/JuliaLang/julia/pull/31466>
>
> Suppose we have a package that does this:
> \`\`\`julia
> module MyPkg
> 
> struct T en…d
> 
> if ccall(:jl\_generating\_output, Cint, ()) == 1
> precompile(setindex!, (Dict{T,Int}, Int, T))
> end
> 
> end
> \`\`\`
> 
> Unfortunately, the \`precompile\` does nothing useful. I dove into this in some detail. It appears that the reason is that no \`setindex!\` methods are defined in \`MyPkg\`, and consequently the corresponding binding \`b\` has \`b-\>value == NULL\`. As a consequence it doesn't get added to the \`\*.ji\` cache file. 
> 
> This is unfortunate, because inferring \`setindex!\` for Dicts is quite expensive. This PR aims to fix that by allowing package authors to write
> 
> \`\`\`julia
> precompile(MyPkg, setindex!, (Dict{T,Int}, Int, T))
> \`\`\`
> 
> Note the module argument in the first slot. This adds it to a list of MethodInstances that should be associated with \`MyPkg\`.
> 
> I've verified it doesn't break anything, but it doesn't yet work. I tried the instructions \[here\](https://docs.julialang.org/en/latest/devdocs/debuggingtips/#Debugging-precompilation-errors-1) but gdb complains
> 
> \`\`\`sh
> (gdb) attach -w -n julia-debug
> Illegal process-id: -w -n julia-debug.
> \`\`\`
> 
> Perhaps someone who knows more about this than me can offer some helpful tips.
> 
> I am not sure this alone will suffice, but I suspect that perhaps in conjunction with just a few other tweaks (and good ways to measure time spent on inference, see #31444 and \[this SnoopCompile branch\](https://github.com/timholy/SnoopCompile.jl/tree/teh/inference\_timing)) we could dramatically cut latencies for certain packages. CC @SimonDanisch, who I know is very interested in the topic.

(Provided there are enough `precompile` calls, presumably generated via SnoopCompile.jl)

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [August 25, 2019, 2:50am UTC](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956/165 "2019-08-25T02:50:05Z")

</div>

That’s definitely something that would be needed to fix this more directly. Without it, you need to develop around it.

---

<div class="post-metadata">

**Author:** ![cce](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cce/32/460_2.png) [@cce](https://discourse.julialang.org/u/cce)\
**Post date:** [August 25, 2019, 9:04am UTC](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956/167 "2019-08-25T09:04:28Z")

</div>

The important measure for me is “time to first response for a lambda container”. As Julia is finding more and more use in an web-service computational setting, it’s important that we have a handle on the startup latency for bringing new computational back ends on-line. Precompilation may help somewhat, but, in my case each new query also causes compilation (which, is why it runs fast). I’d rather have it slow start than a slow finish… but a faster start would be excellent.

---

<div class="post-metadata">

**Author:** ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Post date:** [August 25, 2019, 9:09am UTC](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956/168 "2019-08-25T09:09:30Z")

</div>

> [@baggepinnen](#):
>
> to make sure you do not have to restart Julia

i often wonder about this, once i run a function once. it gets compiled. theoretically we can save the compiled version reload it. the problem is of course when you hace every functuob then the image becomes so bloated.

if you look at the list from NOV 2018. The top 2 is done, sort of. so the emphasis will be on tttp soon. Then it will things like package compilers.

 ![Screenshot_20190825_190114](https://global.discourse-cdn.com/julialang/original/3X/9/5/95a1f102a5893107f4f3a0ef12a935887a69588b.jpeg)

---

<div class="post-metadata">

**Author:** ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Post date:** [August 25, 2019, 9:13am UTC](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956/169 "2019-08-25T09:13:41Z")

</div>

> [@Mason](#):
>
> find myself hoping that a package compiler style solution is found instead of putting all the focus on just reducing compile times in general.

This is such an interesting aspect of Julia. Julia seems unique amongst languages in that we need to talk about trade-offs in compilation vs run time like that!

Just a comment.

---

<div class="post-metadata">

**Author:** ![piever](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/piever/32/1815_2.png) [@piever](https://discourse.julialang.org/u/piever)\
**Post date:** [August 25, 2019, 12:32pm UTC](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956/170 "2019-08-25T12:32:05Z")

</div>

> [@xiaodai](#):
>
> if you look at the list from NOV 2018. The top 2 is done, sort of. so the emphasis will be on tttp soon. Then it will things like package compilers.

I’d like to emphasize that even though threads here sometimes focus on the negative side of things (i.e. time to first plot is still slow), the fact that the compiler team had such an ambitious priority list and managed to deliver on the first half already is impressive. At least in my experience, compiler bugs were somewhat common in Julia 1.0.0 (I hit some doing simple things, esp. with `Missing`) and I haven’t encountered any in a while in later releases. Similarly, multithreading made amazing progress in 1.3. Let’s hope that the work on the remaining items is equally successful!

---

<div class="post-metadata">

**Author:** ![pistacliffcho](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pistacliffcho/32/8518_2.png) [@pistacliffcho](https://discourse.julialang.org/u/pistacliffcho)\
**Post date:** [August 25, 2019, 4:18pm UTC](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956/171 "2019-08-25T16:18:20Z")

</div>

> [@ChrisRackauckas](#):
>
> If it doesn’t infer, don’t put abstract types over things. Handling calculations with “how abstract it should be” is one of the slowest things. Just make inference call it Any.

I’m not saying this advice is _wrong_, but it is a little concerning. If this is just some advice to follow until things are further optimized, that’s fine. But if this is fundamental to compiling, I’d say this is some rather core functionality.

Same with using dictionaries.

---

<div class="post-metadata">

**Author:** ![Zach\_Christensen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/zach_christensen/32/7220_2.png) [@Zach\_Christensen](https://discourse.julialang.org/u/Zach_Christensen)\
**Post date:** [August 25, 2019, 5:03pm UTC](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956/172 "2019-08-25T17:03:07Z")

</div>

I believe he’s referring to dictionaries that store multiple types. This is problematic for inference. This may be a point where the compiler should “give up”.

---

<div class="post-metadata">

**Author:** ![tobias.knopp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tobias.knopp/32/7551_2.png) [@tobias.knopp](https://discourse.julialang.org/u/tobias.knopp)\
**Post date:** [August 26, 2019, 6:25am UTC](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956/173 "2019-08-26T06:25:53Z")

</div>

> [@ChrisRackauckas](#):
>
> The solution is shown to more concretely be:
> 
> 1. Get rid of the KW nonsense
> 2. Concretize the argument handling earlier has to happen, otherwise you already take a 7 second hit
> 3. Concretization of the types should happen by the display function, since 13 seconds of it is type inference into the (hard coded and definitely not runtime loaded) gr\_display function. A large part of this is because the superlinear behavior of the compiler.
> 4. Get rid of Measures.jl. It has too much type information and causes issues. If anything, replace it with values and associated symbols, and hardcode branches instead of over dispatching. Units are just not a good idea for compile time… this would likely be helpful for Gadfly as well.
> 5. Throw a few declarations on the types in the structs that cannot be well inferred, but you know what they are by the time you dig a few functions down. This should only be a few pieces, not every argument that comes out of a KW.

These are very interesting observations. I started a question here:

> [@Does compile time depend on type stability](https://discourse.julialang.org/t/does-compile-time-depend-on-type-stability/6548):
>
> Hi, I have a larger Gtk.jl application that requires more than one minute to start up. If I press some buttons, the initial compile time is also quite high. However, if I run the application a second time everything is totally snappy. So runtime is absolutely ok. I was wondering if it actually helps when I try to make code type stable. So is type stable code faster to compile?

and

> [@Performance Issue with Gtk.jl](https://discourse.julialang.org/t/performance-issue-with-gtk-jl/6171):
>
> Hi, I have a performance issue that may be linked to Gtk.jl. My application needs 30 sec to start up the first time and profiling tells me that most of the time is spend in inference. The app is a little larger so that some dynamic typing (e.g. when using Dicts) cannot be avoided. The inference call is triggered from this line in Gtk.jl https://github.com/JuliaGraphics/Gtk.jl/blob/master/src/GLib/signals.jl#L11 it involves a cfunction so it may be linked to [Performance regression of Cuba.jl …](https://discourse.julialang.org/t/performance-regression-of-cuba-jl-on-julia-0-6/2424)

and

> [@Compiler Performance](https://discourse.julialang.org/t/compiler-performance/11307):
>
> Don’t know if this is the correct category but maybe: Since Julia 0.6 has been released I have seen major performance issues with Julia. After analyzing the issue a little bit I found out that it seems the compile performance that has regressed a lot. It seems that during the transition from 0.5 to 0.6 this has not been observed and I cannot really find deeper discussions on this subject. I now have some examples the regressed and I wonder if there is some “compiler benchmark suite” where the …

ad never got the answer that type inference itself is the problem. Would it be possible taking @ChrisRackauckas suggestions and putting them into the “performance tipps” section of the manual?

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [August 26, 2019, 6:49am UTC](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956/174 "2019-08-26T06:49:31Z")

</div>

That’s an interesting suggestion. Most (all?) of the [Performance Tips](https://docs.julialang.org/en/v1/manual/performance-tips) are about _runtime_ performance, and hints about _compilation performance_ could be useful. However, it is mostly a moving target.

---

<div class="post-metadata">

**Author:** ![tobias.knopp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tobias.knopp/32/7551_2.png) [@tobias.knopp](https://discourse.julialang.org/u/tobias.knopp)\
**Post date:** [August 26, 2019, 7:40am UTC](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956/175 "2019-08-26T07:40:21Z")

</div>

> [@Tamas\_Papp](#):
>
> However, it is mostly a moving target.

As is runtime performance (-\> small union optimization). I understand perfectly that the focus was on runtime in the past, but code patterns that help the compiler would be very very helpful. My Gtk application needs more than 60 seconds to load and clicking a button the first time requires more than 15 seconds. So getting some hints is extremely appreciated.

---

<div class="post-metadata">

**Author:** ![dlfivefifty](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlfivefifty/32/1959_2.png) [@dlfivefifty](https://discourse.julialang.org/u/dlfivefifty)\
**Post date:** [August 29, 2019, 11:10pm UTC](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956/176 "2019-08-29T23:10:54Z")

</div>

> [@ChrisRackauckas](#):
>
> Type-stable code is fine for run time and compile time speed

A big issue is that this is very difficult to debug. It also glosses over other issues, such as large union types (e.g. `StridedArray`) which cause significant compile time issues, even if type-stable.

Compile-time optimisation feels a lot like Matlab optimisation, where there is no intuitive “mental model” of what makes code fast, and so one is left with a bag-of-tricks. This is one of the reasons I always avoided Matlab and why I like Julia.

Compile-caching seems like a better place to focus effort. If pulled off, spending any time on compile time improvements will be rendered useless. Of course this won’t help with non-compiled code, but occasional compiling is much better than constant compiling.

---

<div class="post-metadata">

**Author:** ![asprionj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/asprionj/32/6856_2.png) [@asprionj](https://discourse.julialang.org/u/asprionj)\
**Post date:** [August 31, 2019, 11:04pm UTC](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956/177 "2019-08-31T23:04:57Z")

</div>

I agree:

> [@Mason](#):
>
> I’d much rather an approach that just lets me re-use or even share compile work

> [@cce](#):
>
> The important measure for me is “time to first response for a lambda container”. As Julia is finding more and more use in an web-service computational setting, it’s important that we have a handle on the startup latency for bringing new computational back ends on-line.

IMO, compile time itself does not (really) matter. For interactive REPL work, there’s Revise and the Juno cycler, and waiting some seconds here and there does even help you take one step back, collecting thoughts, take a sip of coffee to relax.

In “production setups”, things get a bit more complicated. There’s many scenarios where you just want to be able to execute a script (with arguments) or just execute a single command by `julia ...` without the need of maintaining a “worker pool” (i.e. Julia sessions having run through a massive `startup.jl` which one would have to set up manually), managed / called via e.g. [tmux](https://tmux.github.io/)…

> [@baggepinnen](#):
>
> the author of PackageCompiler.jl made it very clear in his juliacon talk that he is not interested in maintaining the repository (understandably)

Sad to hear this… really like the basic idea of the package, esp. `compile_incremental` and derived work such as [Fezzik.jl](https://github.com/TsurHerman/Fezzik). For me, this actually solves the “two-language problem”. Because when I have to apply at least one other language / tool / framework / … to make Julia’s runtime performance applicable in production scenarios (or to share my executable stuff with non-Julians), I do have (at least) two “languages”.

Could there be a way of “dumping” the entire state of a REPL (automatically or on user’s demand), like a snapshot? I think to remember having read somewhere that this is not esily done in Julia for several reasons though…

---

<div class="post-metadata">

**Author:** ![cce](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cce/32/460_2.png) [@cce](https://discourse.julialang.org/u/cce)\
**Post date:** [August 31, 2019, 11:40pm UTC](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956/178 "2019-08-31T23:40:29Z")

</div>

> [@asprionj](#):
>
> IMO, compile time itself does not (really) matter.

In my case actual compile time is a critical component of user responsiveness in a production setting. Our query system uses transparent structures for performance and hence the first time that a particular query is encountered it will cause a compilation. Since queries are constructed dynamically on the client side via a user interface, most queries will be unique. One work around is to have a parameterized query mechanism, but in my nominal customer use case, this isn’t an option. I hope this explains.

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [August 31, 2019, 11:57pm UTC](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956/179 "2019-08-31T23:57:46Z")

</div>

Wait… You allow users to execute arbitrary functions?.. Building arbitrary expression isn’t a problem if all underlying functions that get called are compiled

---

<div class="post-metadata">

**Author:** ![dlfivefifty](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlfivefifty/32/1959_2.png) [@dlfivefifty](https://discourse.julialang.org/u/dlfivefifty)\
**Post date:** [September 1, 2019, 7:29am UTC](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956/180 "2019-09-01T07:29:48Z")

</div>

Anything that allows anonymous functions will require fresh compilation. Many packages have use cases like this: Plots, ApproxFun, … Sometimes the issue can be mitigated by wrapping the function in another type.

Maybe we can recap the state of things. There are four ways forward to help with this issue:

1. Static compilation. PackageCompiler.jl and Fezzek.jl are great proofs of concept, but fail for some packages. If these were refined and incorporated into Base that would probably solve a large number users use cases, for example writing command line scripts, but there would be compile time anytime there’s a new type or function involved. This also doesn’t help when actively developing a package.
2. Faster compiling. The compiler could apparently be made multithreaded, if this sped up compiling by 4x most complaints would be dropped (eg if BandedMatrices.jl took 1.5s to load instead of 5.5s). However there are also compiler “bugs” where things take up to 30s to load, and command line scripts would still be too slow.
3. Refining current code. By making code type stable compile time is usually 1s which is fine. But this is hard to debug and some code is inherently type unstable. Macros telling blocks to minimally compile / interpret would help.
4. Interpreter. In principle code could be interpreted the first run with compiling done in a background thread, as in JIT in interpreted languages. This would help package developers and script writers a lot, though I imagine it’s difficult to ensure high performance code is not interpreted.

Personally I’d be very happy if any of 1,2 or 4 were completed, though other users have different needs so will probably have stronger preferences. In an ideal world we’d have all 4 but that’s probably being greedy.

---

<div class="post-metadata">

**Author:** ![lobingera](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lobingera/32/211_2.png) [@lobingera](https://discourse.julialang.org/u/lobingera)\
**Post date:** [September 1, 2019, 8:03am UTC](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956/181 "2019-09-01T08:03:54Z")

</div>

> [@dlfivefifty](#):
>
> The compiler could apparently be made multithreaded

Can you share an example of a multithreaded compiler?

[Previous page](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956.md?page=8)

[Next page](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956.md?page=10)