When "if (a<b) x=1/(a-b)" divides by zero

Specific Domains Numerics
5

I really appreciate your diligence here, but my question was far more rhetorical than I think you interpreted it.

Many of the other examples of -ffast-math’s changes to program behavior in that thread previously to my post had been of the form where -ffast-math would wrongly assume that floating point numbers satisfied “basic math” rules like associativity of + and similar re-arrangements. My read of your stance at that point in the conversation had been that you should not rely upon such “implementation details” of the IEEE behavior — you simply code against basic math rules and assume there’s some precision loss.

This example is slightly different. IEEE floating point numbers actually do obey the “basic arithmetic” rule that a > b implies that a - b > 0. I’m really glad you found that example, because it shows that -ffast-math breaks it in two ways. As you’ve demonstrated, flushing subnormals to zero is one way this gets broken during the subtraction. But this can also be broken if the comparison is done with the results of a CPU operation that’s performed at 80-bits precision (since it may not rounding back down to the IEEE format first). Example in C:

#include <math.h>
#include <stdio.h>
#include <stdlib.h>

int main(int argc, char **argv) {
    double a = atof(argv[1]);
    if (0.5 + a > 1.0)
        printf("%.20f > 1.0: true\n", 0.5 + a);
    else
        printf("%.20f > 1.0: false\n", 0.5 + a);
    return 0;
}

$ gcc test.c -o test && ./test 0.5000000000000001
1.00000000000000000000 > 1.0: false

$ gcc -ffast-math test.c -o test && ./test 0.5000000000000001
1.00000000000000000000 > 1.0: true

So: what does a+b > c actually mean? Does it perform both the addition and comparison at higher precision than you might expect? Or does it perform the addition at a lower precision by flushing a subnormal to zero first and then doing the comparison on that? This is a perfect example of how there’s not a consistent rule for -ffast-math — it’s just breaking what it can in order to do whatever happens to be fast on your current architecture.

Note that b might indeed be a threshold value that you’re using as a “robust” solution in anticipating some “loss” of precision. If you happen to test against that threshold differently in two different places in your program and expect them to agree, you’re in for trouble.

I hold that in general it’s simply intractable to “defensively” code against the transformations that -ffast-math may or may not perform. If a sufficiently advanced compiler is indistinguishable from an adversary, then giving the compiler access to -ffast-math is gifting that enemy nukes. That doesn’t mean you can’t use it! You just have to test enough to gain confidence that no bombs go off with your compiler on your system.

7 Likes