Fast math in NASA benchmark

I noticed in this benchmark that the --fast-math GCC option was far more effective in reducing the runtime then the --math-mode=fast Julia option. What do you think is the reason for that?

1 Like

I think it comes down to being fundamentally bad idea, hence:

3 Likes

But to give you a better answer: the biggest speed-up in gcc is in the matrix multiplication benchmark: using -Ofast here would let them reassociate arithmetic operations (to exploit SIMD instructions), and use FMA instructions if they’re available. Julia’s @simd macro and muladd functions will do something similar, without the blatant unsafe-ness of fastmath.

4 Likes

Is there a “reader’s digest” version of why @fastmath is bad?

It’s an under-specified grab-bag of math-breaking the compiler can do everywhere. Some algorithms actually depend upon math working correctly.

5 Likes

One specific problem with it is that it’s recursive. While at first this feels like a good thing (I want all my code fast), the problem is it can end up applying to stuff in Base or other libraries where it is specifically broken.

Relevant issue https://github.com/JuliaLang/julia/issues/36246 [oops… this was already linked]

There was once a very long (non-readers-digest) discussion about this, but this post is fairly self-contained:

Especially if you combine it with the fact that you can pick a collection of 2046 numbers that can sum to just about any number depending upon their ordering:

Base’s code actually depends upon smartly re-ordering summations to improve precision that fast math might break.

1 Like

I filed an issue to alert developers responsible for the NASA software comparison benchmark suite to the inappropriateness of including fast math in the results.

When fast math option is not used, in the Jacobi iteration benchmark
Julia is faster than the C-language implementation. 6.96 s versus 7.51 s.
Tested with WSL 2, Julia 1.6, generic Linux.

3 Likes