Comparison benchmarks (numerical)


#1

Benchmarks are often not very interesting for people doing numerics. I have (actually with the help of two colleagues) tried to make simple but (we hope) interesting numerical benchmarks. We focused on coding and testing numerical algorithms; as an example, consider Gaussian elimination: this is something you will not use in practice, as you will use, say, lapack routines; but Gaussian elimination is a prototype of array manipulation.
We have compared C++, Python, Python+Pythran, Python+Numba (with some variations) on a small set of problems)., ranging from very simple ones (linear combination) to Weno solution of (simple) hyperbolic PDE problems.
But we are not sure that our Julia coding cannot be improved. If someone wants to have a look, to, propose improvements, you are welcome.

Everything is on Github: here

Thierry Dumont, Lyon -France-


#2

Results are behind so many clicks, I copied them for convenience:

Type f g implicit
C+±lambda 1.012 1.007 none
C+±Pointer 0.988 1.004 0.996
Py 16.21 19.43 9.359
Pythran 1.045 1.101 0.759
Numba 1.896 2.758 1.158
Ju 1.408 0.893 0.816

#3

Those are the Callback results. 3 through 8 on the menu on the right show different benchmarks:

I didn’t look at the Julia code, but skimming through the benchmarks, it looks like Julia did quite well.

EDIT: I haven’t actually messed with sparse matrices, but I’m surprised all the languages seemed to hang around 3x slower than C++ for many of those benchmarks. I would have thought runtime is dominated by a sparse matrix library?


#4

There are a lot of missing @inbounds from what I saw in a quick skim.


#5

I think they disabled bounds check in the script with

julia --check-bounds=no -O 3 main.jl

#6

Also, the stiffness data wants to be SArray/MArray. For C++ you tell the compiler about the size of your tensors, while the julia compiler does not get that info. Since the sizes are 2 x 3 x 6, i.e. tiny, this can make a very large difference. (you don’t need to hard-code sizes; a type-unstable function boundary for entering the hot loop is very pragmatic and fast)


#7

I think they disabled bounds check in the script with

julia --check-bounds=no -O 3 main.jl |

Yes, no need in the source code.


#8

Also, the stiffness data wants to be SArray/MArray. For C++ you tell the
compiler about the size of your tensors, while the julia compiler does
not get that info. Since the sizes are 2 x 3 x 6, i.e. tiny, this can
make a very large difference. (you don’t need to hard-code sizes; a
type-unstable function boundary for entering the hot loop is very
pragmatic and fast)

Very interesting.
You can propose a modification (that is to say a pull request) :slight_smile:
t.