# File IO Buffers too small?

**URL:** <https://discourse.julialang.org/t/file-io-buffers-too-small/89931>\
**Category:** Performance\
**Tags:** binaryio, io\
**Created:** [November 8, 2022, 1:36pm UTC](https://discourse.julialang.org/t/file-io-buffers-too-small/89931 "2022-11-08T13:36:52Z")\
**Posts on this page:** 1\
**Showing post:** 2

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [November 8, 2022, 2:38pm UTC](https://discourse.julialang.org/t/file-io-buffers-too-small/89931/2 "2022-11-08T14:38:24Z")

</div>

> [@mwlidar](#):
>
> My current solution is to allocate a large buffer and fill it with a separate thread. But the usage is quite cumbersome with all the extra synchronization handling and the code has to be specially adapted.

You can use [GitHub - JuliaIO/BufferedStreams.jl: Fast composable IO streams](https://github.com/JuliaIO/BufferedStreams.jl).

But indeed, I have also found the performance of `IOStream` to be unacceptable for many IO tasks where you typically read many small values. One issue with `IOStream` is that it takes a lock on every IO operation in order to be thread-safe.

For binary files, I typically resort to memory mapping the file and wrapping it in an IOBuffer.

So instead of

```julia
io = open("some very large file.dat")

```

I do

```julia
using Mmap
io = IOBuffer(Mmap.mmap("some very large file.dat"))

```

---

_[View the full topic](https://discourse.julialang.org/t/file-io-buffers-too-small/89931)._