I read somewhere that time to first plot is now much faster. I tried it on Windows 10 Julia v1.4 and plot(1:3) is sub 5 seconds now which is quite acceptable!
I wonder what happened, I haven’t been following as closely the development, so pointers to GitHub posts and discourse posts would be appreciated. If none, then discussion here would be cool.
There wasn’t 1 magic bullet. Inlining heuristics got better, inference had about 6 improvements, @nospecialize was added to some code generation. Overall there were about a dozen small improvements that really added up.
Yes, there have been several small incremental improvements to compiler performance, and I believe Plots.jl added a bunch of precompile statements that also helped significantly. Even bigger improvements are on the way, for example use -O1 for code in Plots.jl by JeffBezanson · Pull Request #2544 · JuliaPlots/Plots.jl · GitHub. For me that changed @time display(plot(rand(10))) from 19 seconds to 14. We have a couple more improvements still in the pipeline.
Is the improvement just for Plots.jl or across the board? I had heard about time to first plot improvements in 1.4 and was excited to time using Gadfly, but found it was actually slightly slower on 1.4 than 1.3. Though I’m not confident my test setup was ideal (open REPL, time using, repeat). So I could easily believe there were improvements, I just measured wrong.
your timing isn’t quite right. using might slow down if it does more precompilation. You want to time both using and first plot
Most of the improvement comes from adding precompile statements to a package, and adding @optlevel 1 (see linked PR for the exact syntax) to a package. So it’s not really across the board (yet). I highly recommend adding optlevel 1 to any non-performance-critical code. The setting is per-module, so you can group together code that still needs advanced optimizations if necessary.
I don’t know much about compilers.
Suppose I have a Package “A” w/ three functions f1(), f2(), f3().
@time using A
@time f1()
@time f2()
@time f3()
“First-plot” isn’t only “much alleviated”, it can be superfast, with gnuplot wrapper:
$ time ~/julia-1.6.0-DEV-8f512f3f6d/bin/julia -O1 --compile=min --startup-file=no # -e "using Gaston; plot(rand(10))"
julia> @time using Gaston
0.123084 seconds (54.59 k allocations: 4.599 MiB) # 0.18 sec. in 1.4.0
julia> @time plot(rand(10)) # yes, not using display(plot ...), as showing the time you have to wait, and still it's only 0.39 seconds that way
0.050220 seconds (28.88 k allocations: 1.764 MiB)
Plots.jl got better, in 1.4, for many reasons, even faster in 1.5 (or 1.6) where you get one of the speed-options from above automatically (in some cases -O0 is ok/better option). I opened an issue for the other, than has given me (for other code) 3.7x faster than default (but it can also be much slower).
it says gnuplot not available on my system. seems like not packaged perfectly on Windows? let me search and submit an issue.
Gaston.jl does not include gnuplot, you need to install it yourself; see gnuplot - Browse /gnuplot/5.2.8 at SourceForge.net
I don’t have the bandwidth right now to figure out how to include gnuplot in Gaston (but pull requests are welcome), and it’s easy enough to install.
Gotcha. Need to use the new artifacts system I presume? Anyway, for me, this is a kind of friction that stops me from adopting Gaston.jl.
Testing out gnuplot and figuring out that I might need to set up esoteric PATH scares me so it’s way down the list of things to try.
I also get why devs do not have time for it though. Objectively speaking, making it easy to install (so frictionless to adopt) would be the fastest way to increase adoption. But it’s a trade-off between time-vs-desire-for-wider-adoption which may favour time at this stage for the devs.
Use Linux, it is much easier
Yeah. Not possible for 99% of ppl working in corporations like banks. insurance.
So that’s keeping out alot of potential users.
I am not sure about this, eg see the banks here:
Eh… Obviously banks use Linux in the IT department for servers etc. By most employees aren’t IT… So yeah… This demonstrates a big problem. A bank is made up of so many users and most are locked down by IT and uses only Windows. I suspect who never worked inside big corporations don’t see that and fail to grasp it.
Only specialist functionsc can use Linux and most data science ppl I know at banks use Windows. So dial down my guesstimate to 90%. 99% is probably too big an exaggeration.
I fully grasp it, it is just not my problem. It is a restriction these corporations impose on their own employees. It is within their power to change it.
OS choice is a business decision like any other. Companies should be free to make bad choices, but this should not be anything the FOSS world is concerned with.
While I understand that some people prefer Windows (familiarity), banning qualified people from using Linux or OS/X is like insisting that your employees wear blindfolds and walk backwards on company premises. They will inevitably bump into things and each other, and while it may be mildly amusing, it is not something that anyone else should care about.
It is more about politics. Most IT people know that Linux is much faster and safer but they prefer Windows for corporate use. When something goes wrong, they can blame Microsoft. It is for the same reason that so many banks still use SAS.
I am not sure I understand how that is useful. One can surely blame Microsoft, but this per se does not lead to any tangible benefit (ie bugfixes, free support, or heaven forbid, compensation for damages).
If you are suggesting a principal-agent problem where the employers of IT are clueless and are willing to take “it’s Microsoft’s fault” as an explanation/excuse for any problem, these days that is becoming a caricature of the past at best.
In any case, the target audience of Julia is an expert programmer-scientist with high value added (this includes students who are training to become one). Constraining the choice of OS for such people is like deploying a nuclear submarine in a bathtub.
The expression I usually hear is “no one was ever fired for buying IBM”, i.e., sticking with the mainstream solution is the safe risk-averse career move.
I’d guess that just about anyone using Julia at a bank has enough freedom to also be using Linux. Nearly every quantitative trading and research team I know of uses Linux, typically with a python and C++ stack and maybe Java or kdb+/q mixed in.
Well regarding using Windows, I build HPC clusters for a living but my desktop is Windows.
Let us think of WSL 2 - does anyone have experience of this with Julia?
I cannot test sadly as my laptop is at a lower release of Windows which Windows Subsystem for Linux 2 does not support.