Fast math in NASA benchmark

[mbauman]

There was once a very long (non-readers-digest) discussion about this, but this post is fairly self-contained:

Accurate summation algorithm

A point I’ve made several times previously is the following: Mathematically, since + is associative and commutative, all correct summation algorithms are algebraically equivalent to each other, in particular they are all equivalent to left-to-right summation. The -ffast-math option gives the compiler license to transform any algorithm into any algebraically equivalent one in the mathematical sense. Therefore -ffast-math gives the compiler license to transform any summation algorithm int…

Especially if you combine it with the fact that you can pick a collection of 2046 numbers that can sum to just about any number depending upon their ordering:

Array ordering and naive summation

One of the first things you learn in numerical analysis is that floating-point operations are not associative. A classic example is this: julia> (0.1 + 0.2) + 0.3 0.6000000000000001 julia> 0.1 + (0.2 + 0.3) 0.6 I was thinking of ways to make floating-point summation independent of the order of the summands without making the performance much worse (this is a hobby of mine). Julia currently uses a pairwise summation algorithm, which is much better than naive left-to-right reduction while havin…

Base’s code actually depends upon smartly re-ordering summations to improve precision that fast math might break.