# File IO Buffers too small?

**URL:** <https://discourse.julialang.org/t/file-io-buffers-too-small/89931>\
**Category:** Performance\
**Tags:** binaryio, io\
**Created:** [November 8, 2022, 1:36pm UTC](https://discourse.julialang.org/t/file-io-buffers-too-small/89931 "2022-11-08T13:36:52Z")\
**Posts on this page:** 1\
**Showing post:** 11

<div class="post-metadata">

**Author:** ![mkoculak](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkoculak/32/28310_2.png) [@mkoculak](https://discourse.julialang.org/u/mkoculak)\
**Post date:** [November 11, 2022, 9:42pm UTC](https://discourse.julialang.org/t/file-io-buffers-too-small/89931/11 "2022-11-11T21:42:41Z")

</div>

This is really interesting and might answer why I have observed strongly varying performance while writing data to files - all my tests were done on Windows.

> [@Performance of writing binary files](https://discourse.julialang.org/t/performance-of-writing-binary-files/80278/3):
>
> That was definitely one thing slowing the sequential write! Turning off the lock gave a stable 5x speed-up - making it a reasonable solution if one has no other choice. Still, bulk write is much faster for files below 20-50 MB. When I glimpsed into the Base IO code, it seems Julia specializes writing byte arrays directly into a function called unsafe\_write(), as opposed to normal write() in other cases. Both of them make a call to some C function, so this is as far as I can go. slight_smile …

Maybe it has something to do with caching that Windows performs on recently/frequently accessed files? In my reading tests I have tried to compensate for that by manually clearing read files from the cache, but I am not sure if it was successful.

So maybe also for reading there is an optimum batch size for one read call that would bring you closer to max of your drive? I have tested it on 3 different PCs and 7 or 8 drives and the pattern was stable, just different for each drive.  
I on the other hand should repeat those tests on Linux to see if that solves the problem for my use case as well.

---

_[View the full topic](https://discourse.julialang.org/t/file-io-buffers-too-small/89931)._