# Julia programs now shown on benchmarks game website

**URL:** <https://discourse.julialang.org/t/julia-programs-now-shown-on-benchmarks-game-website/17722>\
**Category:** Community\
**Tags:** announcement\
**Created:** [November 19, 2018, 5:44pm UTC](https://discourse.julialang.org/t/julia-programs-now-shown-on-benchmarks-game-website/17722 "2018-11-19T17:44:02Z")\
**Posts on this page:** 20\
**Page:** 7

<div class="post-metadata">

**Author:** ![non-Jedi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/non-jedi/32/3645_2.png) [@non-Jedi](https://discourse.julialang.org/u/non-Jedi)\
**Post date:** [August 23, 2019, 6:21pm UTC](https://discourse.julialang.org/t/julia-programs-now-shown-on-benchmarks-game-website/17722/122 "2019-08-23T18:21:25Z")

</div>

Ah. I didn’t click through and didn’t realize you were speaking of the speed graph; assumed you were talking about the table under “How source code size is measured” here: [How programs are measured (Benchmarks Game)](https://benchmarksgame-team.pages.debian.net/benchmarksgame/how-programs-are-measured.html)

---

<div class="post-metadata">

**Author:** ![Juan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juan/32/7657_2.png) [@Juan](https://discourse.julialang.org/u/Juan)\
**Post date:** [August 23, 2019, 7:08pm UTC](https://discourse.julialang.org/t/julia-programs-now-shown-on-benchmarks-game-website/17722/123 "2019-08-23T19:08:09Z")

</div>

Are you using the original timings from [Which programming language is fastest?](https://benchmarksgame-team.pages.debian.net/benchmarksgame/) or the new ones from [https://github.com/JuliaPerf/BenchmarksGame.jl](https://github.com/JuliaPerf/BenchmarksGame.jl) ?

---

<div class="post-metadata">

**Author:** ![Karajan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/karajan/32/8545_2.png) [@Karajan](https://discourse.julialang.org/u/Karajan)\
**Post date:** [August 23, 2019, 7:20pm UTC](https://discourse.julialang.org/t/julia-programs-now-shown-on-benchmarks-game-website/17722/124 "2019-08-23T19:20:27Z")

</div>

Ah, no. That table got me wondering but it’s so old Julia isn’t even in there…

> Are you using the original timings […]

Yes, I’m using the timings from the benchmarks website. The “new” ones are actually much older (last update was ~8 months ago) so the data I used represent Julias capabilities better (and more favorably).

---

<div class="post-metadata">

**Author:** ![Jean\_Michel](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jean_michel/32/8282_2.png) [@Jean\_Michel](https://discourse.julialang.org/u/Jean_Michel)\
**Post date:** [August 23, 2019, 8:30pm UTC](https://discourse.julialang.org/t/julia-programs-now-shown-on-benchmarks-game-website/17722/125 "2019-08-23T20:30:13Z")

</div>

I noticed something looking at the Chapel code for pidigits: the author of the Chapel translation really  
understood the algorithm, and made a much clearer description that in most other languages. The Julia translation was made without any effort to understand the algorithm: it shows!

---

<div class="post-metadata">

**Author:** ![non-Jedi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/non-jedi/32/3645_2.png) [@non-Jedi](https://discourse.julialang.org/u/non-Jedi)\
**Post date:** [August 23, 2019, 8:37pm UTC](https://discourse.julialang.org/t/julia-programs-now-shown-on-benchmarks-game-website/17722/126 "2019-08-23T20:37:56Z")

</div>

Ha! You caught me. In my defense, I did make an attempt to understand the algo from the paper linked in the benchmark description, but I didn’t have time (or inclination honestly) to learn enough Haskell to interpret the paper.

---

<div class="post-metadata">

**Author:** ![Jean\_Michel](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jean_michel/32/8282_2.png) [@Jean\_Michel](https://discourse.julialang.org/u/Jean_Michel)\
**Post date:** [August 23, 2019, 8:40pm UTC](https://discourse.julialang.org/t/julia-programs-now-shown-on-benchmarks-game-website/17722/127 "2019-08-23T20:40:27Z")

</div>

Well, you should just then translate the Chapel code to Julia: you will get a better program!

---

<div class="post-metadata">

**Author:** ![non-Jedi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/non-jedi/32/3645_2.png) [@non-Jedi](https://discourse.julialang.org/u/non-Jedi)\
**Post date:** [August 23, 2019, 11:00pm UTC](https://discourse.julialang.org/t/julia-programs-now-shown-on-benchmarks-game-website/17722/128 "2019-08-23T23:00:30Z")

</div>

That’s exactly what I did.

---

<div class="post-metadata">

**Author:** ![non-Jedi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/non-jedi/32/3645_2.png) [@non-Jedi](https://discourse.julialang.org/u/non-Jedi)\
**Post date:** [August 24, 2019, 3:48am UTC](https://discourse.julialang.org/t/julia-programs-now-shown-on-benchmarks-game-website/17722/129 "2019-08-24T03:48:40Z")

</div>

Since you have the data, any chance you could make a list/table of each Julia program sorted by gzipped length relative to the shortest program for each benchmark?

---

<div class="post-metadata">

**Author:** ![Karajan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/karajan/32/8545_2.png) [@Karajan](https://discourse.julialang.org/u/Karajan)\
**Post date:** [August 24, 2019, 7:03am UTC](https://discourse.julialang.org/t/julia-programs-now-shown-on-benchmarks-game-website/17722/130 "2019-08-24T07:03:08Z")

</div>

Of course, I’m just not quite sure what you are asking for exactly? 10 tables, one for each benchmark, only Julia programs (but all of them), sorted by factor of gzipped length vs. shortes length?

If you prefer, I can also clean up my messy script and provide that.

Edit: here we go, the [script](https://gist.github.com/karajan9/3beeeff3c639e959e09aa166140927ce) to get the data and create the plot.

---

<div class="post-metadata">

**Author:** ![ninjaaron](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ninjaaron/32/6392_2.png) [@ninjaaron](https://discourse.julialang.org/u/ninjaaron)\
**Post date:** [August 28, 2019, 9:18pm UTC](https://discourse.julialang.org/t/julia-programs-now-shown-on-benchmarks-game-website/17722/131 "2019-08-28T21:18:05Z")

</div>

> [@non-Jedi](#):
>
> Ha! You caught me. In my defense, I did make an attempt to understand the algo from the paper linked in the benchmark description, but I didn’t have time (or inclination honestly) to learn enough Haskell to interpret the paper.

I’m way out of my depth in this conversation in general, but I wanted to step in for just this moment and say that Haskell is a very nice, very fun language to learn and use. It’s not as pragmatic as Julia (… or, like, any other language), but learning it is a real treat. Programming in general is meat and potatoes (and some of it is boring veggies). Haskell is dessert.

---

<div class="post-metadata">

**Author:** ![Olof\_Salberger](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/olof_salberger/32/4850_2.png) [@Olof\_Salberger](https://discourse.julialang.org/u/Olof_Salberger)\
**Post date:** [December 2, 2019, 1:14am UTC](https://discourse.julialang.org/t/julia-programs-now-shown-on-benchmarks-game-website/17722/132 "2019-12-02T01:14:46Z")

</div>

The Julia version on the website got updated to 1.3 . Unless I managed to gaslight myself, this made the benchmarks run a bit faster: ![Toy benchmark program busy seconds compared to the fastest for selected programming language implementations.](https://benchmarksgame-team.pages.debian.net/benchmarksgame/download/fastest.svg)

Julia is now listed ahead of Java, which is an impressive achievement. The only remaining GCed language which is still ahead of Julia on the benchmarks game is C#.

---

<div class="post-metadata">

**Author:** ![gasagna](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gasagna/32/1275_2.png) [@gasagna](https://discourse.julialang.org/u/gasagna)\
**Post date:** [December 2, 2019, 8:36pm UTC](https://discourse.julialang.org/t/julia-programs-now-shown-on-benchmarks-game-website/17722/133 "2019-12-02T20:36:01Z")

</div>

A few months back I run some of the C and Julia benchmarks on my laptop. Excluding compilation time, Julia was much faster than what shown on the website and typically on par with C. I wonder what would happen to your plot if you removed compilation time, i.e. running code twice, after a first “warm-up”.

---

<div class="post-metadata">

**Author:** ![non-Jedi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/non-jedi/32/3645_2.png) [@non-Jedi](https://discourse.julialang.org/u/non-Jedi)\
**Post date:** [December 2, 2019, 8:47pm UTC](https://discourse.julialang.org/t/julia-programs-now-shown-on-benchmarks-game-website/17722/134 "2019-12-02T20:47:17Z")

</div>

Including jit warmup time is an intentional design decision of the benchmark game. It hurts languages like Java as well. Many of the tasks are compute-intensive enough that Julia’s compilation time doesn’t matter much, but not all are. Julia’s high rank including compilation I think testifies strongly to the strength of Julia’s approach.

---

<div class="post-metadata">

**Author:** ![Ratingulate](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ratingulate/32/9242_2.png) [@Ratingulate](https://discourse.julialang.org/u/Ratingulate)\
**Post date:** [December 2, 2019, 8:53pm UTC](https://discourse.julialang.org/t/julia-programs-now-shown-on-benchmarks-game-website/17722/135 "2019-12-02T20:53:05Z")

</div>

If that’s the case they should include compilation time for the AOT languages.

---

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [December 2, 2019, 9:39pm UTC](https://discourse.julialang.org/t/julia-programs-now-shown-on-benchmarks-game-website/17722/136 "2019-12-02T21:39:57Z")

</div>

There’s some sense in their approach in that with the AOT languages you can just compile a binary once and run it multiple times and even distribute it. For Julia, JIT compilation has to happen again every time you quit and restart julia.

However, perhaps this means that there’s room for us to pull a stunt with PackageCompiler where we do a static step, building a custom sysimage for the BenchmarksGames tests with all methods precompiled.

---

<div class="post-metadata">

**Author:** ![Seif\_Shebl](https://avatars.discourse-cdn.com/v4/letter/s/eada6e/32.png) [@Seif\_Shebl](https://discourse.julialang.org/u/Seif_Shebl)\
**Post date:** [December 3, 2019, 2:19am UTC](https://discourse.julialang.org/t/julia-programs-now-shown-on-benchmarks-game-website/17722/137 "2019-12-03T02:19:41Z")

</div>

Dropping the compilation time can be a bit misleading even for AOT languages. Languages like C++ and Rust move some computations to compile time, this places them in front of C and Fortran in some benchmarks, which is definitely not true. Also, compilation time in Julia seems mostly constant, regardless of the input size, which is really great.

---

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [December 3, 2019, 3:07am UTC](https://discourse.julialang.org/t/julia-programs-now-shown-on-benchmarks-game-website/17722/138 "2019-12-03T03:07:01Z")

</div>

> [@Seif\_Shebl](#):
>
> Languages like C++ and Rust move some computations to compile time

We are quite capable of this in Julia as well. Compile time computations are incredibly important for high performance computing in general and I see no reason why it should be regarded as cheating the benchmarks.

---

<div class="post-metadata">

**Author:** ![igouy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/igouy/32/8524_2.png) [@igouy](https://discourse.julialang.org/u/igouy)\
**Post date:** [December 3, 2019, 6:24am UTC](https://discourse.julialang.org/t/julia-programs-now-shown-on-benchmarks-game-website/17722/139 "2019-12-03T06:24:37Z")

</div>

> Languages like C++ and Rust move some computations to compile time

Please be specific — which programs ?

---

<div class="post-metadata">

**Author:** ![igouy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/igouy/32/8524_2.png) [@igouy](https://discourse.julialang.org/u/igouy)\
**Post date:** [December 3, 2019, 6:25am UTC](https://discourse.julialang.org/t/julia-programs-now-shown-on-benchmarks-game-website/17722/140 "2019-12-03T06:25:15Z")

</div>

> We are quite capable of this in Julia as well.

Please don’t.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [December 3, 2019, 8:58am UTC](https://discourse.julialang.org/t/julia-programs-now-shown-on-benchmarks-game-website/17722/141 "2019-12-03T08:58:40Z")

</div>

> [@Mason](#):
>
> pull a stunt with PackageCompiler where we do a static step, building a custom sysimage for the BenchmarksGames tests with all methods precompiled.

I think that this is considered cheating by the rules of the game.

Also, I am not sure if the Julia community really needs to prove anything so desperately at this point that rigging this game should even be considered (assuming we could, which I doubt). I think it is widely recognized that this is just a _game_, not a competition to pick the “fastest” language.

[Previous page](https://discourse.julialang.org/t/julia-programs-now-shown-on-benchmarks-game-website/17722.md?page=6)

[Next page](https://discourse.julialang.org/t/julia-programs-now-shown-on-benchmarks-game-website/17722.md?page=8)