Weird floating point vs rational behavior

[stevengj]

@albheim

I’m also a bit curious what the printing is actually based on, is it just picking the shortest representation that is closer to the actual value than the neighbouring float values, or what is the rule for printing?

For Float64/32/16, it’s printing the shortest decimal value that rounds to the same value (when rounded to the nearest binary floating point number).

In particular, it uses a state-of-the-art algorithm called Ryū. (Previously it used another algorithm called Grisu that did the same thing.) In particular, as described in the Ryū paper, the criteria for display are:

as laid out in a classic 1990 paper.

See also these posts:

• Working with machine precision - #9 by kristoffer.carlsson
• Are BigFloats unable to exactly represent 0.1? - #6 by Sukera
• Sum of float64 vector gives slightly incorrect answer - #20 by stevengj

and many others… (Note, however, that BigFloat printing uses a separate algorithm implemented in the GNU MPFR library, and IIRC it is not always the shortest decimal representation.)