There wasn’t 1 magic bullet. Inlining heuristics got better, inference had about 6 improvements, @nospecialize was added to some code generation. Overall there were about a dozen small improvements that really added up.
Yes, there have been several small incremental improvements to compiler performance, and I believe Plots.jl added a bunch of precompile statements that also helped significantly. Even bigger improvements are on the way, for example https://github.com/JuliaPlots/Plots.jl/pull/2544. For me that changed @time display(plot(rand(10))) from 19 seconds to 14. We have a couple more improvements still in the pipeline.
Is the improvement just for Plots.jl or across the board? I had heard about time to first plot improvements in 1.4 and was excited to time using Gadfly, but found it was actually slightly slower on 1.4 than 1.3. Though I’m not confident my test setup was ideal (open REPL, time using, repeat). So I could easily believe there were improvements, I just measured wrong.
Most of the improvement comes from adding precompile statements to a package, and adding @optlevel 1 (see linked PR for the exact syntax) to a package. So it’s not really across the board (yet). I highly recommend adding optlevel 1 to any non-performance-critical code. The setting is per-module, so you can group together code that still needs advanced optimizations if necessary.
“First-plot” isn’t only “much alleviated”, it can be superfast, with gnuplot wrapper:
$ time ~/julia-1.6.0-DEV-8f512f3f6d/bin/julia -O1 --compile=min --startup-file=no # -e "using Gaston; plot(rand(10))"
julia> @time using Gaston
0.123084 seconds (54.59 k allocations: 4.599 MiB) # 0.18 sec. in 1.4.0
julia> @time plot(rand(10)) # yes, not using display(plot ...), as showing the time you have to wait, and still it's only 0.39 seconds that way
0.050220 seconds (28.88 k allocations: 1.764 MiB)
Plots.jl got better, in 1.4, for many reasons, even faster in 1.5 (or 1.6) where you get one of the speed-options from above automatically (in some cases -O0 is ok/better option). I opened an issue for the other, than has given me (for other code) 3.7x faster than default (but it can also be much slower).
Gotcha. Need to use the new artifacts system I presume? Anyway, for me, this is a kind of friction that stops me from adopting Gaston.jl.
Testing out gnuplot and figuring out that I might need to set up esoteric PATH scares me so it’s way down the list of things to try.
I also get why devs do not have time for it though. Objectively speaking, making it easy to install (so frictionless to adopt) would be the fastest way to increase adoption. But it’s a trade-off between time-vs-desire-for-wider-adoption which may favour time at this stage for the devs.
Eh… Obviously banks use Linux in the IT department for servers etc. By most employees aren’t IT… So yeah… This demonstrates a big problem. A bank is made up of so many users and most are locked down by IT and uses only Windows. I suspect who never worked inside big corporations don’t see that and fail to grasp it.
Only specialist functionsc can use Linux and most data science ppl I know at banks use Windows. So dial down my guesstimate to 90%. 99% is probably too big an exaggeration.
I fully grasp it, it is just not my problem. It is a restriction these corporations impose on their own employees. It is within their power to change it.
OS choice is a business decision like any other. Companies should be free to make bad choices, but this should not be anything the FOSS world is concerned with.
While I understand that some people prefer Windows (familiarity), banning qualified people from using Linux or OS/X is like insisting that your employees wear blindfolds and walk backwards on company premises. They will inevitably bump into things and each other, and while it may be mildly amusing, it is not something that anyone else should care about.
It is more about politics. Most IT people know that Linux is much faster and safer but they prefer Windows for corporate use. When something goes wrong, they can blame Microsoft. It is for the same reason that so many banks still use SAS.
I am not sure I understand how that is useful. One can surely blame Microsoft, but this per se does not lead to any tangible benefit (ie bugfixes, free support, or heaven forbid, compensation for damages).
If you are suggesting a principal-agent problem where the employers of IT are clueless and are willing to take “it’s Microsoft’s fault” as an explanation/excuse for any problem, these days that is becoming a caricature of the past at best.
In any case, the target audience of Julia is an expert programmer-scientist with high value added (this includes students who are training to become one). Constraining the choice of OS for such people is like deploying a nuclear submarine in a bathtub.
The expression I usually hear is “no one was ever fired for buying IBM”, i.e., sticking with the mainstream solution is the safe risk-averse career move.
I’d guess that just about anyone using Julia at a bank has enough freedom to also be using Linux. Nearly every quantitative trading and research team I know of uses Linux, typically with a python and C++ stack and maybe Java or kdb+/q mixed in.