I have an application which writes a csv as an output but never holds the complete csv in memory.
I would like to use a library such as CSV.jl to
a) take advantage of optimizations (CSV.jl is super fast for reading!)
b) use things like escaping logic
However CSV.jl implements a row by row reading method with CSV.rows but only CSV.write to write a complete file as far as I can see.
Is there any way to do this?
There is an
append kw arg that you can use to add to an existing file.
Is this what you are looking for?
This PR has been merged into the latest CSV.jl version 0.6.2, but I have not tried it yet.
The RowWriter does not seem quite what I need. Given a struct implementing the Tables.jl interface it returns an iterator of the strings which would be written to the file.
The append flag is more in the direction of what I need, although I don’t expect it to be very efficient.
I’ll try the append flag and maybe using a Channel-Tables Interface to insert a row as soon as it is there and write if I find anything good.