`reinterpret` to a single value from an array of a smaller data type

General Usage
21

I did have a look at the source

Is there a way to setup two position pointers into an IOBuffer, ie for separate read/write locations without the seeks? mark could kind of work, or a switchmark would be good, eg (seek the current mark as new pos + store current pos as the new mark)

  function switchmark(io::T) where T<:IO ##Adapted from reset()
    ismarked(io) || (io.mark = 0); ##throw(ArgumentError("$(T) not marked"))
    m = io.mark
    p = io.ptr-1; ##offset of 1
    seek(io, m)
    io.mark = p # must be after seek, or seek may fail
    return m
    end
22

Yeah, but those donā€™t do me much good for random access (unless there is something Iā€™m missing).

23

seek / skip / position?

I think only as PipeBuffer or append=true (one writer at the end position, and the reader independent)

24

I think the problem is that the need to always ultimately resort to read and thereby move the position in the IO buffer every time you read from it is just way more complicated than simply representing it as a reinterpreted array. Clearly I need to look into this more thoroughly though, I may have rejected that solution too quickly when I first got into this.

I definitely tended to have it in my mind that read actually deletes data from the buffer, but I suppose that is never true. Iā€™m definitely too ignorant about this and need to benchmark a prototype that uses only IO and no unsafe methods.

25

I think it would be nice to have a non-allocating, fast read(io, Float64) (e.g.) in Base.

Iā€™m aware of read!(s::IO, x::Ref{T}) where {T}, but the Ref is still a little cumbersome to work with, because at least right now (on master) creating the Ref allocates (which is why read(io, Float64) allocates). So in order to achieve zero allocation, you have to have a cache that contains the pre-allocated Ref.

In the past, Iā€™ve used https://github.com/BioJulia/BufferedStreams.jl to get a read(io, ::Type{Float64}) that doesnā€™t allocate:

https://github.com/BioJulia/BufferedStreams.jl/blob/d7806bd418cbfe7516e18583ec6dd1758cf35d40/src/bufferedinputstream.jl#L190-L204

i.e. what ScottPJones was talking about. Maybe having this kind of thing in Base for certain frequently-used bitstypes would be good. Or maybe we just wait for compiler improvements that elide the Ref allocation.

2 Likes