Parquet2.jl 0.2.25 outputting corrupted files (0.2.26 fix now up)

ExpandingMan
Tue, 16 Jul 2024 13:53:08 +0000
Data

Recently there has been a change in the parquet thrift metadata schema which I included in Parquet2.jl as of 0.2.25. Unfortunately it looks like this is now outputting corrupt files in many circumstances (can confirm that arrow can’t read it). Since Parquet2.jl can read itself fine it seems to suggest that there is something wrong with my implementation of the schema but so far I have been unable to find the discrepancy.

Please avoid 0.2.25. If I am unable to fix this I will have to roll it back. I will get to it as soon as I am able. One way or another I will make another tag by end of day.

6 posts - 3 participants