Need example of processing csv file in chunks

asyrov · January 4, 2021, 5:10pm

I have big 10+GB files which I need to load and process line by line. I’m trying to use CSV package, but appears this package preload entire file.

Can anyone help or point to example of code which works with big files?

marc · January 4, 2021, 5:12pm

This might help:
Reading-Chunks

Skoffer · January 4, 2021, 5:29pm

If you want to process data row by row, then maybe Rows can be useful? Home · CSV.jl

mthelm85 · January 4, 2021, 5:38pm

Have a look at JuliaDB too.