Julia programs now shown on benchmarks game website

announcement

#42

With regards to using packages, the benchmarks are so simple there isn’t much point to packages imo, and there is value in having completely self contained examples that can be copy pasted.

Of course one could use e.g StaticArrays.jl for the nobody benchmark but showing how one can create their own static vector in a few lines shows off julia more than hiding it behind a package imo.


#43

See also my Julia needs work at "Benchmark Game": numbers seem off up to 100x slower, maybe "Julia AOT" entries needed?


#44

Aren’t the programs in Kristofer’s repo in good shape? I believe some or all of them should be submitted. I guess I could (but should I)? I don’t want to take anyone’s credit.

I decided to look into one of the programs that didn’t have a Julia implementation (I see however now it’s in Kristofer’s repo), and to translate this fastest [C++] non-multi-threaded version:

https://benchmarksgame-team.pages.debian.net/benchmarksgame/program/spectralnorm-gpp-2.html

It uses Eigen library, and since that’s allowed, I guess we could use similar [Julia] libraries, or even the Julia Eigen wrapper.

However, I just started with translating the commented out “EQUIVALENT MATLAB IMPLEMENTATION”. It’s trivial to do, but I opted to try the http://sciencecow.mit.edu/matlab-to-julia/ translator anyway. I ended up with this (maybe it just isn’t “EQUIVALENT”, anyone know what’s wrong?) that’s fast on my very old laptop but wrong:

julia> function approximate(n)
       A = zeros(n,n)
         for i=1:n
           for j=1:n
             A[i,j] = 1.0/((i+j)*(i+j+1)/2.0 + i+1.0)
           end
         end
         u = ones(n,1)
         v = zeros(n,1)
         for i=1:10
           v = A'*(A*u)
           u = A'*(A*v)
         end
         sqrt((u'*v)/(v'*v))
       end
approximate (generic function with 1 method)

julia> @time approximate(5500)
  5.461645 seconds (9.13 k allocations: 232.940 MB, 0.72% gc time)
1×1 Array{Float64,2}:
 0.361967

The translator got this line wrong (anyone know why? and I should file a bug):

sqrt((u"*v)/(v"*v))

and I changed integers to floats here: A[i,j] = 1.0/((i+j)*(i+j+1)/2.0 + i+1.0)

I got the same result as expected and seems same speed (was just checking).

I opened um my MATLAB clone, Octave to just make sure, and I get the same number eventually. Probably after minutes, ten[s?]. At least Julia is way faster, can anyone compare real MATLAB to Julia for me?


#45

You access the matrix in a suboptimal pattern. Try switching the order of the for loops


#46

Thanks, I assumed they had good MATLAB code, and I didn’t look into row vs. column major (I assumed they did differently, though stating “EQUIVALENT”).

Still of course I get the same wrong result.

This however doesn’t change the timing for Julia at least. And for Octave I gave up waiting.

For lower number/quicker I get about 6 to 7 seconds either way for 550 (sometimes the “better” version is faster sometimes the other).

While in Julia I get:

julia> @time approximate(550)
  0.049661 seconds (70 allocations: 2.491 MB)
1×1 Array{Float64,2}:
 0.361967

#47

There is also a “Recursive Fibonacci Benchmark using top languages on Github”.

It would be nice to do it better.


#48

From HN"ok, owner of the repo here. So this project was purely to show the macro differences between interpreted ruby and compiled crystal to beginner ruby devs at a meet up.

I didn’t expect to get a memoized version of every language and really don’t think comparing them from a performance benchmark makes much sense."

fyi The same comparison was removed from the benchmarks game and replaced with tasks that were still toy but more than a dozen lines.