Time to first plot benchmark

General Usage
,
1

In the light of Julia survey result (i.e people complain about time to the first plot), I did a naive benchmark, as expected, PyPlot is the fastest :frowning:

http://uaf-8.t2.ucsd.edu/~jling/dump/PlotBenchmark.html

Because it takes some time to display the image (@time will appear way before image actually shows up), I chose to do put the timing like this which is a bit hacky – you need to hit restart kernel and run all cells.

To my surprise, Gadfly is slower than Plots.

1 Like
2

You should add the line

t = time(); Plots.GR.histogram(a);println("Plain GR used $(time() - t)s")

to measure the plain GR performance. :smile:

2 Likes
3

good point, updated! GR is pretty fast :wink:

4

for fair comparison, now they have the same nbins=20

5

… but you are still measuring some Plots overhead. The first time to plot (with plain GR) should be about 2-3 seconds.

6

where is the Plots overhead?

7

Not sure, but could you try

julia> @time using GR
  0.183892 seconds (83.67 k allocations: 5.665 MiB)

julia> @time histogram(randn(10000), nbins=50)
  3.965902 seconds (8.77 M allocations: 442.108 MiB, 5.76% gc time)

on your machine. My laptop is old and slow … :disappointed_relieved:

The above result are Julia 1.3.0-alpha.1 on macOS Catalina 10.15.

8

I noticed in Jupyter there’s delay between cell calls, fixed that, still ~4s.

The thing is, @time does NOT reflect the time it takes for you to see the plot, you can try this in Jupyter notebook. when @time spits out the answer, you haven’t seen the plot yet.

julia> @time using GR  
0.194347 seconds (159.49 k allocations: 10.088 MiB)

julia> @time histogram(randn(10000), nbins=50)  
3.863596 seconds (10.47 M allocations: 524.561 MiB, 6.45% gc time)

julia> versioninfo()
Julia Version 1.1.1
Commit 55e36cc308 (2019-05-16 04:10 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)
Environment:
  JULIA_NUM_THREADS = 7
9

Ok, then it’s probably better to perform the benchmarks in a simple shell environment …

10

Then how does that reflect real world speed?

11

try @time display(plot(...))

1 Like
12

That does work, but Gadfly would open a new tab instead of doing it inline. The time measured that way is close to the current ones.