'using Plots' takes 20 seconds or more

I’ve read that calls to using Plots can sometimes be slow but I wonder if as slow as I’m seeing is normal. On 3 trials after starting Julia from the command line I @timed this taking between 35 and 40 seconds.

After some Googling, I learned how to build my system image. After doing that I’m getting more like 19.7 seconds so that definitely makes a difference but 20 seconds still seems awfully long.

Also of note, using -j 2 tends to approximately double the time to run the using command.

Is there something I’m missing or is this just par for the course?

Julia Version 0.6.1
Commit 0d7248e (2017-10-24 22:15 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin14.5.0)
  CPU: Intel(R) Core(TM) i5-4258U CPU @ 2.40GHz
  WORD_SIZE: 64
  uname: Darwin 16.7.0 Darwin Kernel Version 16.7.0: Wed Oct  4 00:17:00 PDT 2017; root:xnu-3789.71.6~1/RELEASE_X86_64 x86_64 i386
Memory: 8.0 GB (605.25 MB free)
Uptime: 1.069672e6 sec
Load Avg:  2.2373046875  2.3759765625  2.30517578125
Intel(R) Core(TM) i5-4258U CPU @ 2.40GHz: 
       speed         user         nice          sys         idle          irq
#1  2400 MHz     625768 s          0 s     393018 s    2275837 s          0 s
#2  2400 MHz     329560 s          0 s     164461 s    2800507 s          0 s
#3  2400 MHz     620082 s          0 s     329063 s    2345384 s          0 s
#4  2400 MHz     340340 s          0 s     168468 s    2785719 s          0 s

It’s a known problem.

1 Like

Using master of Plots.jl is faster as it is precompiled, but still slow:

julia> @time using Plots                                                                                                                                                 
  6.168379 seconds (4.04 M allocations: 219.459 MiB, 1.31% gc time)                                                                                                      
                                                                                                                                                                         
julia> @time plot(1:3)                                                                                                                                                   
 16.983929 seconds (10.69 M allocations: 569.126 MiB, 1.96% gc time)                                                                                                     

then it still takes another 6s or so for the plot to show (pyplot backend). I use PyPlot.jl because I can’t be bothered to wait; that takes about 6s to first plot.

You should be able to make it fast (<1s) with a custom userimg. But for that you need to make sure all the important functions get compiled. Have a look at the readme of SnoopCompile.

1 Like

I should write a tutorial on how to use Plotly,js with Pages.js. There is an undocumented Plotly.js example in Pages.jl (master). It is super fast and interactive :+1:

Edit: I should qualify “super fast” with this group :sweat_smile: I did not compare to other packages so it was more of an absolute statement about general satisfaction with the speed. I wouldn’t try anything super complicated and expect great performance. There are other great packages that are probably faster the plotly :slight_smile:

1 Like

I’ve been working on this problem in MakiE, which is a new design prototype for Plots, with only the glvisualize backend for now.
I just did the following benchmarks:

Plotting one million points in 3s start to end :slight_smile:

The trick is to make your package precompile save and use snoop compile, to emit precompile statements to tell julia what functions to compile binary for, when included in the system image.

I will create a package and instructions for what I did to make loading fast! This could be part of Pkg.build().

19 Likes

Whaaat??? This looks like a game-changer!

[EDIT: for comparison, something like this takes like 4.7 seconds in Mathematica (3,3GHz Core i7)]

Mathematica used to take 30 seconds to start up when I last used it. (Old iMac.) perhaps it’s preloading the Plot[] code :grinning:

1 Like

Of course, it’s 4.7 seconds after starting Mathematica (v11), which makes this comparison all the more striking. I really hope Simon is able to keep the performance at this level as he continues to flesh out MakiE. @sdanisch, is it reasonable to expect that the libuserimg.so approach could remain transparent to the user, and indeed make it part of the Pkg.build() process?

For Makie I plan to have a Pkg.build() process, which can be steered by environment variables for now.
I will see @StefanKarpinski thinks about integration into Pkg3. From all I heard, the plan is to improve caching of compiled binary for Julia in 1.0, so maybe this will become a natural part of Julia anyways.

4 Likes

That would be awesome! It really pains me every time google spits back at me that old article complaining about precompilation times in Julia.

4 Likes

Does this problem solved now?
2020/04/14

As of 1.4, I get 6 seconds (after precompile), so not fully fixed yet but about 3x better. It’s still improving though. 1.4 is better than 1.3, and 1.5 will be better than 1.4 was.

3 Likes

really good news.

Gnuplot.jl (master branch, new release due in a few days) takes ~5 seconds with the default qt terminal, and no precompilation or modified system image:
Screenshot_20200415_195448.

but uses gnuplot as backend…

4 Likes

Just one more data point… Gaston master (precompiled):

$ time julia -e 'using Gaston; plot(randn(10^6),randn(10^6))'

real	0m1.000s
user	0m1.168s
sys	0m1.379s
4 Likes

I can confirm that both Gaston and Gnuplot are very capable. And FAST. I highly recommend this to anyone who is looking for fast plotting. Knowledge of gnuplot is a plus, of course, but for someone programming in Julia that shouldn’t be a problem.

3 Likes

Plots has good speed on the second plot as we all know, unlike the (slight) second-plot issue I discovered with another package:

https://github.com/mbaz/Gaston.jl/issues/133

There’s no real need to do this redundantly (might still happen in real-world code), but using only gets stable (allocations/speed) on third use (unlike for Gaston, where it only applies to the plot):

julia> @time using Plots
  9.869769 seconds (7.44 M allocations: 422.704 MiB, 2.78% gc time)

julia> @time using Plots
  1.492128 seconds (1.70 M allocations: 80.019 MiB, 1.90% gc time)

julia> @time using Plots
  0.000298 seconds (276 allocations: 14.859 KiB)
1 Like

Is this issue still actively worked on?

@time using Plots
11.354988 seconds (18.48 M allocations: 1.024 GiB, 3.34% gc time)

$ time julia -e “using Plots”

real 0m11,482s
user 0m10,651s
sys 0m0,373s

$ time python -c “import matplotlib”

real 0m0,868s
user 0m0,411s
sys 0m0,264s

$ time gnuplot -e “”

real 0m0,026s
user 0m0,023s
sys 0m0,003s

I thought Julia wanted to rival C’s speed, not be slower than python.

1 Like

What version are you on? On Master, I’m seeing 3 seconds, which while not solved, is a massive improvement. (time julia -e “using Gaston” also clocks in at just over 1 second for me)

1 Like

I’m using the latest version that Arch Linux ships: julia version 1.5.3

$ time julia -e “using Gaston”

real 0m1,743s
user 0m1,580s
sys 0m0,146s