ANN: BufIO: New I/O interfaces for Julia

Package Announcements
, ,
1

I’m pleased to announce a new package BufIO.jl.

Overview of BufIO.jl

BufIO provides new and improved I/O interfaces for Julia inspired by Rust, and designed around exposing buffers to users in order to explicitly copy bytes to and from them. Compared to the Base.IO interface, the new interfaces in this package are:

  • Lower level
  • Faster
  • Easier to reason about
  • Better specified, with more well-defined semantics
  • Free from slow fallback methods that silently trash your performance

Beside the new interfaces, BufIO also provides a small set of basic types to make use of the new interface, and/or allow easy interoperation between Base.IO types and the new buffered interfaces.

The new types include:

  • BufReader <: AbstractBufReader: A type that wraps a Base.IO to provide the new AbstractBufReader interface
  • BufWriter <: AbstractBufWriter: A type that wraps a Base.IO to provide the new AbstractBufWriter interface
  • CursorReader <: AbstractBufReader: Wrap any contiguous memory-backed bytes in a stateful reader
  • IOReader <: Base.IO: A type that wraps an AbstractBufReader and provides the Base.IO interface
  • VecWriter <: AbstractBufWriter: A faster and simpler alternative to IOBuffer usable e.g. to build strings.

Comparison to other packages

The packages BufferedStreams.jl and TranscodingStreams.jl also provide IO wrapper types that buffer their wrapped io. However, both these packages do so as a transparent optimisation, whereas BufIO.jl provides a different interface.

History of BufIO.jl

I’ve been writing a bunch of different parsers in Julia for about five years. Each time, I’ve found it necessary to create a reader type containing a buffer, then read into the buffer, and then do all my actual parsing on the buffer. From what I hear from others, that’s also how they do it. Somehow, after years of doing that, I didn’t put two and two together about what that implied about the lack of performance and convenience Julia IO interface.

A few years ago, I learned Rust. One of the most common Rust performance footguns is doing IO operations on unbuffered data (unlike Julia, Rust does not buffer several of its basic IO types).
On the other hand, Rust has the BufRead trait, which I’ve found extremely well designed and usable. That led me to believe that an IO interface should be buffered by default, and center its API around the buffer.

The penny finally dropped about a year ago, after a few packages forced me to attempt to handle generic Base.IO objects in Julia, which I found to be a miserable experience due to… many, many issues of the poorly designed Base.IO. I tried to push a (backwards compatible) extension to Base’s IO interface, but it’s difficult to make sweeping changes to Base without close collaboration with someone with commit rights. So:

image
image529×395 64.6 KB

Jokes aside, BufIO.jl is my vision for an alternative I/O interface in Julia, which I previously published here. Ideally, the interface should make it to Base, but I’m skeptical that will ever happen. Whether it does or not, it’s useful to prototype the interface in a package to at least be more informed about what a different I/O interface would feel like.

BufIO.jl is not yet in registration, but will be registered soon. I’ve used this interface already in other packages and found it quite expressive and fast.

I’m very much interested in feedback and suggestions for the interface! Please open issues on the repo GitHub - BioJulia/BufIO.jl: Interface for efficient IO in Julia

Notes on the design of the new interface

  • BufIO has distinct AbstractBufReader and AbstractBufWriter types instead of Base’s common IO type. I’m not 100% sure which is better, but I’ve found that most use of IO uses either writing or reading, not both, and I’ve found that separating the two interfaces makes implementations cleaner and less bloated. It’s also very easy to work around and create a reader/writer type T by e.g. creating a function writer(::T)::AbstractBufWriter.

  • The types in BufIO is by default not threadsafe. Locks are slow, and most programs are single threaded. If users want to protect their resources when used concurrently, they’ll need to use a lock themselves. This tradeoff is no different from any other datastructure, such as a Dict which we (correctly) understand in Julia should also not be threadsafe by default.

  • The current BufIO performance is limited by a few limitiations of Base Julia:

    • BufIO’s VecWriter currently uses Base/Core internals to manipulate Vector.
      Ideally, that should be made API, since I don’t think I do anything ill adviced. Anyway, it means BufIO currently use internals which I’m not that happy about.
    • BufIO is hard hit by the compiler limitation in issue 53584 (no ABI for pointer-ful union types). That restriction will probably be lifted soon-ish.
    • BufIO abstracts “chunks of memory” using MemoryRef, but this does not allow zero-allocation reading/writing of strings since you can’t take a MemoryRef to a string. Hopefully that restriction will be lifted in the future.
36 Likes
2

This looks really cool!

Do you have some benchmarks showing how much faster?

3

I haven’t made any, but I can cook up some quickly.
Here, comparing IOBuffer and VecWriter:

julia> function test_write(io)
            for i in 1:100
                write(io, i)
            end
            write(io, [0x01, 0x02, 0x03, 0x04, 0x05])
            write(io, "abcdefghijklmnopq")
            for i in 1.0f0:0.1f0:10.0f0
                 write(io, i)
             end
             io isa IOBuffer ? String(take!(io)) : String(io.vec)
       end

julia> @btime test_write(VecWriter());
  612.495 ns (7 allocations: 4.25 KiB)

julia> @btime test_write(IOBuffer());
  2.487 μs (203 allocations: 6.22 KiB)

And here comparing IOStream with BufReader{IOStream}:

julia> function test_read(io, v::Vector{UInt8})
           empty!(v)
           for i in 1:100
               push!(v, read(io, UInt8))
           end
           for i in 1:10
              push!(v, 0x01)
              readbytes!(io, @views(v[end-1:end]), 1)
           end
           for i in 1:100
               eof(io) && break
               unsafe_read(io, pointer(v, length(v)), UInt(1))
           end
           close(io)
           return v
       end

julia> @btime test_read(BufReader(open("src/base.jl")), UInt8[]);
  4.039 μs (14 allocations: 8.88 KiB)

julia> @btime test_read(open("src/base.jl"), UInt8[])
  7.604 μs (11 allocations: 816 bytes)

julia> @btime test_read(open("src/base.jl"; lock=false), UInt8[]);
  4.340 μs (11 allocations: 816 bytes)

Note that the latter test heavily favors IOStream because:

  1. BufReader is a layer of indirection over IOStream, and so requires allocating a superfluous buffer and double-copying. If IOStream exposed the AbstractBufReader interface directly, it would be faster than either.
  2. The greatest performance speed comes when you elect to not use the Base APIs that tend to allocate but instead operate on the buffer directly and use functions like read_into!. This makes it hard to do an apples-to-apples comparison (e.g. using BufferedStreams in the benchmark above would probably yield the same results as BufReader)
8 Likes