Arrow, Feather, and Parquet

tkf
Fri, 13 Sep 2019 20:43:18 +0000
Data

Continuing the discussion from JDF - an experimental DataFrame serialization format is ready for beta testing:

@ExpandingMan Thank you very much! That’s a great summary and it helps me getting idea of reliability level of feather format. It’s also nice to get a summary of the recent activities in Arrow community.

JDF - an experimental DataFrame serialization format is ready for beta testing

I think what they are getting at is that Feather was basically deprecated in favor of another arrow file format,

Reading that Parquet and Arrow C++ communities joining forces last year, I was expecting they were going to merge on-disk file format. Do you know why they are keeping two file formats? Maybe because with arrow format you can just mmap it? (And it’s not possible with parquet?)

49 posts - 17 participants