Does Debian's BenchmarkGames show representative performance?

A.
Debian’s benchmark game doesn’t allow showing Julia’s fastest speed, but its fork does (or so far only enabled for a few benchmarks, I convinced the guy to allow julia/aot, and thanks @kristoffer.carlsson for PackageCompiler.jl), and there Julia/aot is 3rd, ahead of Rust and C, only 9% slower than fastest entry C++:

What does it take to eliminate the 117ms “time(sys)” (fully or mostly), as with the other languages? Then Julia would be 19% faster than the current top entry.

https://github.com/hanabi1224/Programming-Language-Benchmarks/issues/291?notification_referrer_id=NT_kwDOAHonKLI0MTg4NDEyNjcyOjgwMDU0MTY&notifications_query=reason%3Aparticipating#issuecomment-1216960434

Without AoT, Julia is only 14th, 41% slower than fastest (but not really, since Julia’s fixed overhead is misleading).

B.
I would still like Julia faster without AoT, the extra 174 ms are compilation overhead that might be possible to reduce. For scripting (and benchmarking), a separate non-default sysimage, e.g. stripping out LinearAlgebra might be useful. I’ve been asked if any alternative sysimage is official, so could Julia include a such JuliaLite sysimage, that would just be taking up space until you enable it?

3 Likes