[ANN] JDF.jl - Experimental Julia DataFrames serialization format

Thanks for sharing this! Found from google while looking for the equivalent of pandas.DataFrame.to_hdf, pandas.read_hdf, xarray.Dataset.to_netcdf, etc in Julia. It would be nice to have interop compatibility with other languages via HDF5, which seems to be a fairly standard way to serialize tables. As a bonus, makes it easy to do chunking & blosc:zstd compression, which is necessary for serializing large data structures.

Seems there’s an issue for this already: Opening Pytables/Pandas file · Issue #92 · JuliaIO/HDF5.jl · GitHub.

Python and R already have decent DataFrame interop. It’d be great to be able to have interop with Julia, too!

edit: just read more about Parquet, looks like good interop across R, Python & Julia. But sounds like writing from Julia is still a bit painful. It looks like Arrow (backend for Feather) 1.0.0 is just around the corner, though! Introducing Apache Arrow Flight: A Framework for Fast Data Transport | Apache Arrow and stability guarantees here: arrow/Versioning.rst at master · apache/arrow · GitHub