Public, allocation-free way to write a `Float64` as `print` does?

General Usage
1

I’m writing large XML files (XLSX.jl’s streaming writer) and need to write millions of Float64s into an IO or a byte buffer. The text has to be what print gives: the shortest decimal that reads back as the same value, e.g. 0.1 rather than 0.10000000000000001.

print(io, x) produces the right text, but it allocates on every call:

julia> io = IOBuffer(); f(io, xs) = foreach(x -> print(io, x), xs);

julia> xs = rand(10^5); f(io, xs); take!(io);

julia> @allocated(f(io, xs)) / length(xs)   # bytes per float
454.1

That’s about 450 bytes per float, and in our benchmark it was a large share of the total allocation.

So far, I’ve looked at:

  • Base.Ryu.writeshortest(buf, pos, x) - This does exactly what I want, writing into a caller’s buffer without allocating. But it’s internal: Base.ispublic(Base.Ryu, :writeshortest) == false. Also, I’m not yet fully convinced it covers the necessary range of floats Excel uses.
  • Printf.format(io, Printf.Format("%.17g"), x) - This is public, but it allocates even more (about 500 B) and doesn’t give the shortest round-trip form.
  • Parsers.jl - Perhaps unsurprisingly, these only parse!

For now, I have a small workaround of our own for floats with up to 15 significant digits. It checks that m / 10^d == x using exact float division and falls back to print otherwise. It works (allocations down to ~30 bytes per float), but it’s a partial reimplementation of something Base already does.

So:

  1. Is there a public API (in Base, a stdlib, or a package) that writes a float’s shortest representation into an IO or buffer without allocating?
  2. If not, is Base.Ryu.writeshortest stable and wide enough in practice to rely on, or would it be reasonable to propose making it public?

Thanks!

2

It’s not Base’s public, but just in case, have you checked BufferIO.jl?

using BufferIO
io = BufWriter(IOBuffer())
write(io, 5.0)

See this post from the package author using integers.