I would like to test Julia’s support for Apache Arrow with a big file that doesn’t fit to memory (RAM). I can’t find any example of such file, could you help me with this please?
I guess I need a file that is in Arrow IPC Format (Feather file format), version 2 (see for example Feather File Format — Apache Arrow v3.0.0).
I would like to read the file into a dataframe like this:
df = DataFrame(Arrow.Table(the_big_file)), as exemplified here: User Manual · Arrow.jl
Thanks for creating Julia