# Fast copy from stream to IOBuffer?
**URL:** https://discourse.julialang.org/t/fast-copy-from-stream-to-iobuffer/42099
**Category:** Performance
**Created:** [June 26, 2020, 1:17pm UTC](https://discourse.julialang.org/t/fast-copy-from-stream-to-iobuffer/42099 "2020-06-26T13:17:25Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@bobcassels](https://discourse.julialang.org/u/bobcassels)
#### Post date: [June 26, 2020, 1:17pm UTC](https://discourse.julialang.org/t/fast-copy-from-stream-to-iobuffer/42099/1 "2020-06-26T13:17:25Z")
I need to read parts of a (binary) stream, and concatenate them, so I can read from that concatenated buffer. I’m reconstructing a thing that was packetized into segments.
I see I can do `write(iob, read(s, ...))`, but I am afraid that will do two copies and an intermediate allocation. Allocating the result of the `read`, copying data once to the result, then copying it again to the IOBuffer. Is there a way to copy directly from the stream to the IOBuffer?
Even better, could the compiler notice and optimize this case so I can write the code above, but the compiler arranges to not materialize the result of the `read`?
---
### Author:  [@Skoffer](https://discourse.julialang.org/u/Skoffer)
#### Post date: [June 26, 2020, 1:46pm UTC](https://discourse.julialang.org/t/fast-copy-from-stream-to-iobuffer/42099/2 "2020-06-26T13:46:57Z")
I guess there are better answers, but if you know size of the binary chunk, you can use [readbytes!](https://docs.julialang.org/en/v1/base/io-network/#Base.readbytes!) function. This way you wouldn’t avoid allocating completely, but it can be irrelevant because it is one time allocation.
Here is an example (untested)
```julia
buf = Vector{UInt8}(undef, chunk_size)
while true
nb = readbytes!(s, buf)
write(iob, @view buf[1:nb])
eof(s) && break
end
```
---
### Author:  [@bobcassels](https://discourse.julialang.org/u/bobcassels)
#### Post date: [June 26, 2020, 5:51pm UTC](https://discourse.julialang.org/t/fast-copy-from-stream-to-iobuffer/42099/3 "2020-06-26T17:51:34Z")
Good idea! That still copies the data twice, but it’s in the right direction…
---
### Author:  [@jameson](https://discourse.julialang.org/u/jameson)
#### Post date: [June 28, 2020, 3:40pm UTC](https://discourse.julialang.org/t/fast-copy-from-stream-to-iobuffer/42099/4 "2020-06-28T15:40:16Z")
`write(iob, s)`?
---
### Author:  [@bobcassels](https://discourse.julialang.org/u/bobcassels)
#### Post date: [June 28, 2020, 3:54pm UTC](https://discourse.julialang.org/t/fast-copy-from-stream-to-iobuffer/42099/5 "2020-06-28T15:54:20Z")
[Back to my original response. Not coherent today…]
I don’t think that helps for my case. I don’t want the whole stream, just the next bytes – the body of the fragmented packet.
---
### Author:  [@xiaodai](https://discourse.julialang.org/u/xiaodai)
#### Post date: [June 29, 2020, 2:30am UTC](https://discourse.julialang.org/t/fast-copy-from-stream-to-iobuffer/42099/6 "2020-06-29T02:30:15Z")
I can think of a way to manage the buffer I write to manually (if I know how big it needs to be). I just need a specialised `write` and `read` method for that manual buffer. I just allocate an array, keep track of where I am up to and use `readbyte!` into that array at the right location.
---
### Author:  [@sairus7](https://discourse.julialang.org/u/sairus7)
#### Post date: [June 29, 2020, 7:06am UTC](https://discourse.julialang.org/t/fast-copy-from-stream-to-iobuffer/42099/7 "2020-06-29T07:06:12Z")
> Input streams instead support the notion of “anchoring”, which instructs the stream to save the current position in the buffer. If the buffer gets refilled, then any data in the buffer including or following that position gets shifted over to make room. When the match is finished, one can then call `takeanchored!` return an array of the bytes from the anchored position to the currened position, or `upanchor!` to return the index of the anchored position in the buffer.
[https://biojulia.net/BufferedStreams.jl/stable/inputstreams.html](https://biojulia.net/BufferedStreams.jl/stable/inputstreams.html)
---
### Author:  [@bobcassels](https://discourse.julialang.org/u/bobcassels)
#### Post date: [September 28, 2020, 12:49am UTC](https://discourse.julialang.org/t/fast-copy-from-stream-to-iobuffer/42099/8 "2020-09-28T00:49:54Z")
Okay. So I want `write(s::IO, from::IO, nb::Integer)`. I’ll implement it, when I get a chance, and submit a PR.
Looks like I can just take the code from `readbytes_some!` (called from `read(s::IO, nb::Integer = typemax(Int))`), and combine that with the code from `write(s::IO, a::Array)`, skipping the intermediate array. It would be okay with me if someone got to it before me. 😉