Help with Arrow.jl and size of files

[bkamins]

@austin-putz

No errors but I’ll try :auto and Int8 or Bool. Yes, CSV is slow, just used to write out to double check the file against my input matrix (think my function is working as I thought it should).

You say “without compression”, so :auto gives me the compression? I thought Arrow did this naturally, but didn’t have time to read their full documentation yet.

If you have no errors without :auto it means that you are on very old DataFrames.jl and I recommend you update it.

:auto does not give you compression. You can turn-on compression when writing file to disk (but most likely it might lead to slower write times, at the benefit of having smaller file size).