# Roadmap for a faster time-to-first-plot?

**URL:** <https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956>\
**Category:** Internals & Design\
**Tags:** ttfp\
**Created:** [April 9, 2019, 1:22pm UTC](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956 "2019-04-09T13:22:42Z")\
**Posts on this page:** 20\
**Page:** 12

<div class="post-metadata">

**Author:** ![dlfivefifty](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dlfivefifty/32/1959_2.png) [@dlfivefifty](https://discourse.julialang.org/u/dlfivefifty)\
**Post date:** [November 30, 2019, 11:43pm UTC](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956/223 "2019-11-30T23:43:35Z")

</div>

I’m a bit confused what’s changed. In Julia v0.5 or so I used SnoopCompile on all tests to generate precompiles and this worked quite well. Then at some point the precompiles stopped showing any noticeable improvement and I deleted the precompiles.

But now you are saying use SnoopCompile To generate precompile statements, but not on the tests?

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [November 30, 2019, 11:53pm UTC](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956/224 "2019-11-30T23:53:13Z")

</div>

> [@dlfivefifty](#):
>
> I’m a bit confused what’s changed. In Julia v0.5 or so I used SnoopCompile on all tests to generate precompiles and this worked quite well. Then at some point the precompiles stopped showing any noticeable improvement and I deleted the precompiles.

It never really changed. It just has to do with how much gets invalidated and how much of it can be isolated to specific packages. Precompiles which “share types” are currently just dropped (probably one of the biggest limitations to even using something like SnoopCompile)

We need to time more, but my feeling is that if Gadfly/Compose and Plots both dropped their dependency on Measurements.jl and instead embedded a version into themselves as a package so that all of those weird number types can precompile and then a snoop file is used for that, I think it could do quite a bit.

(In fact, if anyone wants to contribute, this is a very nice thing to try. Make Measurements.jl a submodule of Plots by pasting it in, and use the SnoopCompile stuff from Leon’s PR to add precompiles that utilize this module. See what happens. Beginner friendly!)

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [December 1, 2019, 3:41am UTC](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956/225 "2019-12-01T03:41:22Z")

</div>

> [@ChrisRackauckas](#):
>
> Make Measurements.jl a submodule of Plots by pasting it in,

I guess this can be done with just a few lines, without copying the whole codebase?

```julia
include(Base.locate_package(Base.PkgId(Base.UUID("eff96d63-e80a-5855-80a2-b1b0885c5ab7"), "Measurements")))
using .Measurements # notice the leading dot

```

(Note that, strictly speaking, this requires Measurements.jl to make `deps` a part of public API.)

---

<div class="post-metadata">

**Author:** ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)\
**Post date:** [December 1, 2019, 3:49am UTC](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956/226 "2019-12-01T03:49:20Z")

</div>

Are you talking about [https://github.com/JuliaGraphics/Measures.jl](https://github.com/JuliaGraphics/Measures.jl)  
?

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [December 1, 2019, 4:00am UTC](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956/227 "2019-12-01T04:00:57Z")

</div>

Yes Measures

---

<div class="post-metadata">

**Author:** ![tim.holy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tim.holy/32/52_2.png) [@tim.holy](https://discourse.julialang.org/u/tim.holy)\
**Post date:** [December 1, 2019, 10:47am UTC](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956/228 "2019-12-01T10:47:01Z")

</div>

> [@ChrisRackauckas](#):
>
> Indeed, but it was coupled with some ways to instrument the compiler and find out precisely what’s taking the most time to compile, and some dives into what’s causing times to grow and discussions of method invalidation

You don’t really have to do any diving now with `@snoopi`. The old `@snoopc` measured codegen time (dominated by LLVM time), which isn’t terribly relevant. With `@snoopi` you both measure inference time and can then figure out how well your intervention “worked.”

---

<div class="post-metadata">

**Author:** ![mkborregaard](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkborregaard/32/556_2.png) [@mkborregaard](https://discourse.julialang.org/u/mkborregaard)\
**Post date:** [December 2, 2019, 8:54am UTC](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956/229 "2019-12-02T08:54:31Z")

</div>

It’s already in a submodule so this should be very easy [https://github.com/JuliaPlots/Plots.jl/blob/master/src/Plots.jl#L149-L164](https://github.com/JuliaPlots/Plots.jl/blob/master/src/Plots.jl#L149-L164)

---

<div class="post-metadata">

**Author:** ![daschw](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/daschw/32/2926_2.png) [@daschw](https://discourse.julialang.org/u/daschw)\
**Post date:** [December 2, 2019, 4:15pm UTC](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956/230 "2019-12-02T16:15:25Z")

</div>

I just tried that and could not notice any significant change.

#### master

```julia
julia> @time using Plots
11.759940 seconds (22.34 M allocations: 1.123 GiB, 3.96% gc time)

julia> @time display(plot(rand(10)))
Plot{Plots.GRBackend() n=1}
14.330170 seconds (28.71 M allocations: 1.385 GiB, 3.41% gc time)

```

#### measures

(with code from Measures.jl copied into a submodule of Plots)

```julia
julia> @time using Plots
 12.408045 seconds (22.58 M allocations: 1.137 GiB, 4.08% gc time)

julia> @time display(plot(rand(10)))
Plot{Plots.GRBackend() n=1}
 14.317090 seconds (28.69 M allocations: 1.384 GiB, 3.63% gc time)

```

---

<div class="post-metadata">

**Author:** ![jheinen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jheinen/32/2787_2.png) [@jheinen](https://discourse.julialang.org/u/jheinen)\
**Post date:** [December 2, 2019, 5:55pm UTC](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956/231 "2019-12-02T17:55:29Z")

</div>

GR now contains a [`snoop`](https://github.com/jheinen/GR.jl/blob/master/examples/snoop.jl) function that can be used to minimize time-to-first-plot:

```julia
julia -e 'using PackageCompiler; compile_incremental("GR", joinpath(pwd(), "snoop.jl"), cc_flags="-mtune=\"generic;sandybridge,-xsaveopt,clone_all;haswell,-rdrnd,base(1)\"")'
time julia snoop.jl

```

The results are OK for me:

```julia
Build shared library "/usr/local/lib/julia/packages/PackageCompiler/CJQcs/sysimg/sys.dylib":

`cc -shared '-DJULIAC_PROGRAM_LIBNAME="/usr/local/lib/julia/packages/PackageCompiler/CJQcs/sysimg/sys.dylib"' -o /usr/local/lib/julia/packages/PackageCompiler/CJQcs/sysimg/sys.dylib -Wl,-all_load /usr/local/lib/julia/packages/PackageCompiler/CJQcs/sysimg/sys.a -std=gnu99 -I/usr/local/Applications/Julia-1.3.app/Contents/Resources/julia/include/julia -DJULIA_ENABLE_THREADING=1 -fPIC -L/usr/local/Applications/Julia-1.3.app/Contents/Resources/julia/lib -Wl,-rpath,/usr/local/Applications/Julia-1.3.app/Contents/Resources/julia/lib -Wl,-rpath,/usr/local/Applications/Julia-1.3.app/Contents/Resources/julia/lib/julia -ljulia -m64 -O3 '-mtune="generic;sandybridge,-xsaveopt,clone_all;haswell,-rdrnd,base(1)"' -Wl,-install_name,@rpath//usr/local/lib/julia/packages/PackageCompiler/CJQcs/sysimg/sys.dylib`

real 0m1.166s
user 0m0.655s
sys 0m0.524s

```

I have a similar script for the `Plots + GR` combination, which I can provide, if desired …

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [December 2, 2019, 6:17pm UTC](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956/232 "2019-12-02T18:17:41Z")

</div>

🤷‍♂️ thanks for trying it!

---

<div class="post-metadata">

**Author:** ![jheinen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jheinen/32/2787_2.png) [@jheinen](https://discourse.julialang.org/u/jheinen)\
**Post date:** [December 2, 2019, 10:02pm UTC](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956/233 "2019-12-02T22:02:06Z")

</div>

There’s a significant change for `Plots` + `GR`:

```julia
% time julia snoop-Plots.jl
julia snoop-Plots.jl 33.29s user 1.16s system 101% cpu 34.065 total

```

… compared with a “package-compiled” `sysimg`:

```julia
% time julia snoop-Plots.jl
julia snoop-Plots.jl 0.92s user 0.34s system 95% cpu 1.323 total

```

[`snoop-Plots.jl`](https://github.com/jheinen/GR.jl/blob/master/examples/snoop-Plots.jl) is a mix of plain `GR` and basic `Plots` functions.

The speedup is ~ 35x.

---

<div class="post-metadata">

**Author:** ![ianfiske](https://avatars.discourse-cdn.com/v4/letter/i/58f4c7/32.png) [@ianfiske](https://discourse.julialang.org/u/ianfiske)\
**Post date:** [December 23, 2019, 3:53pm UTC](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956/234 "2019-12-23T15:53:17Z")

</div>

Wow, this looks promising. Thanks @jheinen! I’m trying it locally, but PackageCompiler seems to have trouble locating shared object libraries:

```julia
julia -e 'using PackageCompiler; compile_incremental("GR", "/Users/ifiske/julia-testing/snoop-GR-Plots.jl", cc_flags="-mtune=\"generic;sandybridge,-xsaveopt,clone_all;skylake,-rdrnd,base(1)\"")'
┌ Warning: seriestype violin has been moved to StatsPlots. To use: `Pkg.add("StatsPlots"); using StatsPlots`
└ @ Plots /Users/ifiske/.julia/packages/Plots/qZHsp/src/args.jl:1079
┌ Warning: seriestype violin has been moved to StatsPlots. To use: `Pkg.add("StatsPlots"); using StatsPlots`
└ @ Plots /Users/ifiske/.julia/packages/Plots/qZHsp/src/args.jl:1079
┌ Warning: seriestype violin has been moved to StatsPlots. To use: `Pkg.add("StatsPlots"); using StatsPlots`
└ @ Plots /Users/ifiske/.julia/packages/Plots/qZHsp/src/args.jl:1079
┌ Warning: seriestype violin has been moved to StatsPlots. To use: `Pkg.add("StatsPlots"); using StatsPlots`
└ @ Plots /Users/ifiske/.julia/packages/Plots/qZHsp/src/args.jl:1079
┌ Warning: seriestype violin has been moved to StatsPlots. To use: `Pkg.add("StatsPlots"); using StatsPlots`
└ @ Plots /Users/ifiske/.julia/packages/Plots/qZHsp/src/args.jl:1079
ERROR: LoadError: LoadError: error compiling #kde: error compiling conv: error compiling *: could not load library "@rpath/libfftw3.3.dylib"
dlopen(@rpath/libfftw3.3.dylib, 1): image not found

```

I’d appreciate any ideas. I’ve experimented with PackageCompiler in quite a few other ways and always run into some similar error, where it cannot find a shared library, though the specific shared library varies by what I’m trying. I’ve tried setting the LD\_LIBARAY\_PATH variable and also passing the `-L` cc\_flag, but this doesn’t help. I’m on OS X, julia 1.3.

---

<div class="post-metadata">

**Author:** ![jheinen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jheinen/32/2787_2.png) [@jheinen](https://discourse.julialang.org/u/jheinen)\
**Post date:** [December 23, 2019, 9:42pm UTC](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956/235 "2019-12-23T21:42:46Z")

</div>

I think the problem is not related to `GR` directly. It’s probably another package dependency (for `FFTW.jl`). On macOS, the latter might be provided by `Homebrew` - did you try to rebuild it?

---

<div class="post-metadata">

**Author:** ![ianfiske](https://avatars.discourse-cdn.com/v4/letter/i/58f4c7/32.png) [@ianfiske](https://discourse.julialang.org/u/ianfiske)\
**Post date:** [December 23, 2019, 10:01pm UTC](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956/236 "2019-12-23T22:01:06Z")

</div>

Thanks for the tip – GR and all associated packages (including FFTW) work fine on my system outside of PackageCompiler. I only get these `image not found` errors from PackageCompiler attempts. Just to check though, I just did `] build FFTW` and then repeated the same experiment above, and got the same error.

---

<div class="post-metadata">

**Author:** ![anon92994695](https://avatars.discourse-cdn.com/v4/letter/a/ce7236/32.png) [@anon92994695](https://discourse.julialang.org/u/anon92994695)\
**Post date:** [December 24, 2019, 5:16pm UTC](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956/237 "2019-12-24T17:16:07Z")

</div>

While I was horking around Julia GUI’s and things I found something. I’m of course very interested in the ideas of interactive plots using native Julia and stuff like that so I ended up in Plots.jl. When I was groking the GR backend for how plots get rendered, I realized something a little funny. It looks like every display has a read and write step via something called GKS? Is that correct? I’m no wizard, but typically reading and writing is pretty slow compared to in memory. I don’t foresee this being a huge deal in time-to-first plot but, it could be if the plots were large enough. Could also introduce latency in animations.

[https://github.com/JuliaPlots/Plots.jl/blob/f2e98acc65c417fce2b8338a3300c9a790f40008/src/backends/gr.jl#L1408](https://github.com/JuliaPlots/Plots.jl/blob/f2e98acc65c417fce2b8338a3300c9a790f40008/src/backends/gr.jl#L1408)

Is there a direct way to convert Plots to a graphical context without I/O? Or does this even matter according to benchmarks?

---

<div class="post-metadata">

**Author:** ![jheinen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jheinen/32/2787_2.png) [@jheinen](https://discourse.julialang.org/u/jheinen)\
**Post date:** [December 30, 2019, 7:51am UTC](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956/238 "2019-12-30T07:51:04Z")

</div>

For `PDF` as shown in the above example it would not make much sense. In `Plots`, however, the `GR.emergencyclosegks()` might cause some overhead within GR, which should be avoided in a future version.

---

<div class="post-metadata">

**Author:** ![tbo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tbo/32/9357_2.png) [@tbo](https://discourse.julialang.org/u/tbo)\
**Post date:** [February 6, 2020, 8:27am UTC](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956/239 "2020-02-06T08:27:56Z")

</div>

The @time macro does not seem to tell the full truth, at least for me. Doing the timing actually gives me 7 seconds.

```julia
julia> @time (using Plots; gr(); plot([1,2,3], [4,5,6]))
  7.053711 seconds (15.98 M allocations: 845.897 MiB, 3.60% gc time)

```

However, at the time the output (7s) is shown, the plot is not yet displayed. It will (measured manually) take an additional ~5-6 seconds for the plot to pop up, i.e. in total it’s 13 seconds not 7. A second plot will however appear nearly instantly. Is this a known behavior and if so, where do these additional 5-6 seconds come from?

---

<div class="post-metadata">

**Author:** ![JeffreySarnoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeffreysarnoff/32/1980_2.png) [@JeffreySarnoff](https://discourse.julialang.org/u/JeffreySarnoff)\
**Post date:** [February 6, 2020, 8:33am UTC](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956/240 "2020-02-06T08:33:36Z")

</div>

What version of Julia are you running (`julia> versioninfo()`)?  
Have you updated your packages (`julia> using Pkg; Pkg.update()`)?

---

<div class="post-metadata">

**Author:** ![mkborregaard](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkborregaard/32/556_2.png) [@mkborregaard](https://discourse.julialang.org/u/mkborregaard)\
**Post date:** [February 6, 2020, 8:35am UTC](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956/241 "2020-02-06T08:35:34Z")

</div>

They come from calling `display` on the `Plot` object, which is implicitly called when returning the plot to the REPL. Call `display(plot(...))` explicitly to time the full operation.

---

<div class="post-metadata">

**Author:** ![tbo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tbo/32/9357_2.png) [@tbo](https://discourse.julialang.org/u/tbo)\
**Post date:** [February 6, 2020, 8:47am UTC](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956/242 "2020-02-06T08:47:30Z")

</div>

Thank’s for the quick response, this is really good to know, when doing some measurements!

[Previous page](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956.md?page=11)

[Next page](https://discourse.julialang.org/t/roadmap-for-a-faster-time-to-first-plot/22956.md?page=13)