# How to read lines from a file with a reusable buffer?

**URL:** <https://discourse.julialang.org/t/how-to-read-lines-from-a-file-with-a-reusable-buffer/58620>\
**Category:** General Usage\
**Created:** [April 5, 2021, 2:43pm UTC](https://discourse.julialang.org/t/how-to-read-lines-from-a-file-with-a-reusable-buffer/58620 "2021-04-05T14:43:58Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![sstadick](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sstadick/32/10776_2.png) [@sstadick](https://discourse.julialang.org/u/sstadick)\
**Post date:** [April 5, 2021, 2:43pm UTC](https://discourse.julialang.org/t/how-to-read-lines-from-a-file-with-a-reusable-buffer/58620/1 "2021-04-05T14:43:58Z")

</div>

What I want to do:

```julia
line_buffer = UInt8[]
buffered_io_stream = open("file.txt")

while readuntil!(line_buffer, buffered_io_stream, '\n') != 0
     # do thing with line_buffer
     # clear line_buffer
end

```

Is there a package out there that does this, ideally into a `Vector{UInt8}`?

Looking at the source code, `readuntil` is delegated to some C code? [julia/iostream.jl at f9720dc2ebd6cd9e3086365f281e62506444ef37 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/blob/f9720dc2ebd6cd9e3086365f281e62506444ef37/base/iostream.jl#L435)

Is there any good documentation on the IO / Stream / Buffered versions of those and how to implement them? All I’m seeing is this section of the docs: [I/O and Network · The Julia Language](https://docs.julialang.org/en/v1/base/io-network/#General-I/O) Which just describes the methods but doesn’t really get into the expected interfaces for something to be a sub-type of `IO` for example.

Lastly, are there plans to improve the standard library with regards to Strings/byte strings and improved IO code written in julia instead of C? From what I’m seeing here: [Milestones - JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/milestones), there are no significant changes planned for Strings or File IO related APIs.

---

<div class="post-metadata">

**Author:** ![sstadick](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sstadick/32/10776_2.png) [@sstadick](https://discourse.julialang.org/u/sstadick)\
**Post date:** [April 5, 2021, 3:00pm UTC](https://discourse.julialang.org/t/how-to-read-lines-from-a-file-with-a-reusable-buffer/58620/2 "2021-04-05T15:00:30Z")

</div>

I’m actually pretty sure that there is no easy way to do this given what exists in the stdlib. I’d have to implement my own buffered reader since there is no function like “fill buffer” on `IOStream`.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [April 6, 2021, 1:07pm UTC](https://discourse.julialang.org/t/how-to-read-lines-from-a-file-with-a-reusable-buffer/58620/3 "2021-04-06T13:07:09Z")

</div>

> [@sstadick](#):
>
> there is no function like “fill buffer” on `IOStream` .

There is `read!(::IO, a::Array{UInt8})`.

---

<div class="post-metadata">

**Author:** ![sstadick](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sstadick/32/10776_2.png) [@sstadick](https://discourse.julialang.org/u/sstadick)\
**Post date:** [April 6, 2021, 2:19pm UTC](https://discourse.julialang.org/t/how-to-read-lines-from-a-file-with-a-reusable-buffer/58620/4 "2021-04-06T14:19:20Z")

</div>

True, but that fills an array I pass it. There is no method to fill the IOStream.ios buffer, which would be needed to implement an alternate read\_until!.

Here is what I’ve tried:

```julia
function readuntil2!(ret::Vector{UInt8}, stream::IOStream, delim::UInt8, keep::Bool=false)
    @assert isreadable(stream)
    resize!(ret, 0)
    buffer1 = stream.ios
    resize!(ret, 0)
    filled = 0
    # Here is where I want to read bytes into the stream.ios buffer, but I don't see any
    # functions that do that
    while read(stream, 4096) > 0
        pos = findfirst(x -> x == delim, buffer1)
        if isnothing(pos)
            sz = length(buffer1)
            if length(ret) < filled + sz
                resize!(ret, filled + sz)
            end
        else
            sz = pos - buffer1.bufferpos + 1
            resize!(ret, filled + sz)
        end
        copyto!(ret, filled + 1, buffer1, sz)
        filled += sz
        if !isnothing(pos)
            break
        end
    end
    return ret
end

# And also
function readuntil3!(ret::Vector{UInt8}, stream::IOStream, delim::UInt8, keep::Bool=false)
    @assert isreadable(stream)
    resize!(ret, 0)
    # This give me a weird error about not being able to determine the return type
    ccall(:jl_readuntil, ret, (Ptr{Cvoid}, UInt8, UInt8, UInt8), stream.ios, delim, 0, !keep)
    nothing
end

```