# \#parquet

**URL:** https://discourse.julialang.org/tag/parquet/317.md

[Latest](https://discourse.julialang.org/latest.md) · [Categories](https://discourse.julialang.org/categories.md) · [Tags](https://discourse.julialang.org/tags.md)

---

## [\[RFC/ANN\] Parquet3.jl — alternative Parquet implementation focused on support for nested columns](https://discourse.julialang.org/t/rfc-ann-parquet3-jl-alternative-parquet-implementation-focused-on-support-for-nested-columns/136295)

<div class="topic-metadata">

**Author:** [@Yuan-Ru-Lin](https://discourse.julialang.org/u/Yuan-Ru-Lin)\
**Replies:** 9\
**Last updated:** [April 10, 2026, 2:59am UTC](https://discourse.julialang.org/t/rfc-ann-parquet3-jl-alternative-parquet-implementation-focused-on-support-for-nested-columns/136295 "2026-04-10T02:59:17Z")

</div>

I’d like to announce and request comments on Parquet3.jl. This implementation focuses specifically on support for nested column types, which are explicitly out of scope for Parquet2.jl, but would be a suitable representa…

---

## [Parquet2.jl](https://discourse.julialang.org/t/parquet2-jl/128639)

<div class="topic-metadata">

**Author:** [@Sam\_P](https://discourse.julialang.org/u/Sam_P)\
**Replies:** 3\
**Last updated:** [May 6, 2025, 10:00pm UTC](https://discourse.julialang.org/t/parquet2-jl/128639 "2025-05-06T22:00:34Z")

</div>

Hi all, just a quick question regarding Parquet2.jl. Love the package and it has been a life-saver more than once. However, when starting a new repo I noticed that it is blocking LazyArrays from updating. LazyArrays v1.…

---

## [Reading data with 0 row groups](https://discourse.julialang.org/t/reading-data-with-0-row-groups/127318)

<div class="topic-metadata">

**Author:** [@phchavesmaia](https://discourse.julialang.org/u/phchavesmaia)\
**Replies:** 0\
**Last updated:** [March 24, 2025, 7:07pm UTC](https://discourse.julialang.org/t/reading-data-with-0-row-groups/127318 "2025-03-24T19:07:45Z")

</div>

Hello everyone, I am trying to work with big data in Julia. I just exported a .parquet file/folder using python’s dask library and wanted to load it in Julia. It follows a representation of my code using Parquet2: Data…

---

## [Converting CSV to Parquet in Julia](https://discourse.julialang.org/t/converting-csv-to-parquet-in-julia/57328)

<div class="topic-metadata">

**Author:** [@oo92](https://discourse.julialang.org/u/oo92)\
**Replies:** 22\
**Last updated:** [August 11, 2024, 1:08pm UTC](https://discourse.julialang.org/t/converting-csv-to-parquet-in-julia/57328 "2024-08-11T13:08:25Z")

</div>

Hi. I have a simple dataframe that I want to convert to parquet. This is my attempt: begin df = CSV.read("/home/onur/julia-assignment/temp.csv", DataFrame) prq = Parquet.File(df) end But this is the error I’m gettin…

---

## [Parquet2.jl 0.2.25 outputting corrupted files (0.2.26 fix now up)](https://discourse.julialang.org/t/parquet2-jl-0-2-25-outputting-corrupted-files-0-2-26-fix-now-up/117102)

<div class="topic-metadata">

**Author:** [@ExpandingMan](https://discourse.julialang.org/u/ExpandingMan)\
**Replies:** 5\
**Last updated:** [July 19, 2024, 6:29pm UTC](https://discourse.julialang.org/t/parquet2-jl-0-2-25-outputting-corrupted-files-0-2-26-fix-now-up/117102 "2024-07-19T18:29:10Z")

</div>

Recently there has been a change in the parquet thrift metadata schema which I included in Parquet2.jl as of 0.2.25. Unfortunately it looks like this is now outputting corrupt files in many circumstances (can confirm th…

---

## [Parquet2jl: ThriftInvalidFinalReadState: while attempting to read Parquet2.Metadata.Statistics](https://discourse.julialang.org/t/parquet2jl-thriftinvalidfinalreadstate-while-attempting-to-read-parquet2-metadata-statistics/116668)

<div class="topic-metadata">

**Author:** [@PavloPolovyi](https://discourse.julialang.org/u/PavloPolovyi)\
**Replies:** 4\
**Last updated:** [July 9, 2024, 10:02pm UTC](https://discourse.julialang.org/t/parquet2jl-thriftinvalidfinalreadstate-while-attempting-to-read-parquet2-metadata-statistics/116668 "2024-07-09T22:02:28Z")

</div>

Got this error when trying to open parquet files converted from arrow and compressed with snappy by js library called parquet-wasm. Opening with pyarrow or r-arrow works fine. Parquet2jl version 0.2.24 (also earlier vers…

---

## [\[ANN\] Parquet2.jl](https://discourse.julialang.org/t/ann-parquet2-jl/69274)

<div class="topic-metadata">

**Author:** [@ExpandingMan](https://discourse.julialang.org/u/ExpandingMan)\
**Replies:** 20\
**Last updated:** [May 8, 2024, 5:30am UTC](https://discourse.julialang.org/t/ann-parquet2-jl/69274 "2024-05-08T05:30:37Z")

</div>

Parquet2.jl A little something I’ve been working on for the past few weeks. This package is not yet registered because I would like to try to get the community’s help with a bit more testing before registering it. Parq…

---

## [Parquet2.jl: type Nothing has no field meta\_data](https://discourse.julialang.org/t/parquet2-jl-type-nothing-has-no-field-meta-data/113613)

<div class="topic-metadata">

**Author:** [@rongcuid](https://discourse.julialang.org/u/rongcuid)\
**Replies:** 6\
**Last updated:** [April 29, 2024, 5:09pm UTC](https://discourse.julialang.org/t/parquet2-jl-type-nothing-has-no-field-meta-data/113613 "2024-04-29T17:09:59Z")

</div>

I am trying to read some data using Parquet2.jl. The data was generated by pyarrow.parquet around 1MB in size, so very small. Compression is Snappy, as by default in pyarrow. I am getting a very long stack trace about t…

---

## [Write Large Parquet to S3](https://discourse.julialang.org/t/write-large-parquet-to-s3/102639)

<div class="topic-metadata">

**Author:** [@mrufsvold](https://discourse.julialang.org/u/mrufsvold)\
**Replies:** 6\
**Last updated:** [August 9, 2023, 8:43pm UTC](https://discourse.julialang.org/t/write-large-parquet-to-s3/102639 "2023-08-09T20:43:09Z")

</div>

I have an AWS Glue table made up of parquet files. Unfortunately, the upstream process creates files that are too small for optimal querying from Athena/Redshift so I am looking to consolidate them up to files closer to …

---

## [Parquet2 RFC: removing \`CacheVector\`](https://discourse.julialang.org/t/parquet2-rfc-removing-cachevector/86903)

<div class="topic-metadata">

**Author:** [@ExpandingMan](https://discourse.julialang.org/u/ExpandingMan)\
**Replies:** 0\
**Last updated:** [September 7, 2022, 5:13pm UTC](https://discourse.julialang.org/t/parquet2-rfc-removing-cachevector/86903 "2022-09-07T17:13:41Z")

</div>

When I wrote Parquet2.jl, I had what at the time I thought was a terribly clever idea about only loading specific subsets of the full buffer. My motivation for this was based on the observation that, while parquet is al…

---

## [KeyError when writing Parquet files to S3](https://discourse.julialang.org/t/keyerror-when-writing-parquet-files-to-s3/79187)

<div class="topic-metadata">

**Author:** [@a.frist](https://discourse.julialang.org/u/a.frist)\
**Replies:** 1\
**Last updated:** [April 8, 2022, 1:09pm UTC](https://discourse.julialang.org/t/keyerror-when-writing-parquet-files-to-s3/79187 "2022-04-08T13:09:53Z")

</div>

I’m trying to write a parquet file to S3 and I’m just not sure exactly what the issue is. This code has worked in the past and there doesn’t seem to be a significant change to the underlying data. Potentially, all the va…

---

## [How to write in .parquet (or any compressed extension)](https://discourse.julialang.org/t/how-to-write-in-parquet-or-any-compressed-extension/67264)

<div class="topic-metadata">

**Author:** [@Caique\_Rodrigues](https://discourse.julialang.org/u/Caique_Rodrigues)\
**Replies:** 3\
**Last updated:** [August 29, 2021, 11:06pm UTC](https://discourse.julialang.org/t/how-to-write-in-parquet-or-any-compressed-extension/67264 "2021-08-29T23:06:20Z")

</div>

I’m doing some simulation that the output can be around 1GB to 10GB, it’s not so big but we are probably going to run lots of simulations. Right now we are saving the output in .txt, but I’d like to write it in a better …

---

## [Displaying a parquet file in Arrow](https://discourse.julialang.org/t/displaying-a-parquet-file-in-arrow/57398)

<div class="topic-metadata">

**Author:** [@oo92](https://discourse.julialang.org/u/oo92)\
**Replies:** 7\
**Last updated:** [March 17, 2021, 7:32pm UTC](https://discourse.julialang.org/t/displaying-a-parquet-file-in-arrow/57398 "2021-03-17T19:32:34Z")

</div>

Hi. I am trying to add a Parquet file into Arrow. I tried to follow the Arrow.jl docs and implement it like this: begin df = CSV.read("/home/onur/julia-assignment/temp.csv", DataFrame) file = file = "/home/onur/julia…

---

## [Arrow, Feather, and Parquet](https://discourse.julialang.org/t/arrow-feather-and-parquet/28739)

<div class="topic-metadata">

**Author:** [@tkf](https://discourse.julialang.org/u/tkf)\
**Replies:** 48\
**Last updated:** [November 1, 2020, 11:02am UTC](https://discourse.julialang.org/t/arrow-feather-and-parquet/28739 "2020-11-01T11:02:16Z")

</div>

Continuing the discussion from JDF - an experimental DataFrame serialization format is ready for beta testing: @ExpandingMan Thank you very much! That’s a great summary and it helps me getting idea of reliability level …

---

## [Struggling to implement Tables.jl interface for Vector{MyStruct}](https://discourse.julialang.org/t/struggling-to-implement-tables-jl-interface-for-vector-mystruct/42318)

<div class="topic-metadata">

**Author:** [@freeman](https://discourse.julialang.org/u/freeman)\
**Replies:** 8\
**Last updated:** [July 2, 2020, 10:38pm UTC](https://discourse.julialang.org/t/struggling-to-implement-tables-jl-interface-for-vector-mystruct/42318 "2020-07-02T22:38:39Z")

</div>

I’m trying to make Vector{MyStruct} a Table. One of the things that I have in mind is that I’ll be able to save Vector{MyStruct} into a parquet file by just calling write\_parquet(file, tbl), which you can see should be p…

---

## [Trying to write a parquet writer. Please help!](https://discourse.julialang.org/t/trying-to-write-a-parquet-writer-please-help/38653)

<div class="topic-metadata">

**Author:** [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Replies:** 1\
**Last updated:** [May 9, 2020, 12:06pm UTC](https://discourse.julialang.org/t/trying-to-write-a-parquet-writer-please-help/38653 "2020-05-09T12:06:15Z")

</div>

I am trying to understand enough about parquet to get a parquet writer going. I am trying to read a parquet file and I am using Thrift.jl. It says at offset 4 there is DictionaryPageHeader so I read that using Thrift.jl…

---

## [Trying to read parquet file that is using Zstd codec](https://discourse.julialang.org/t/trying-to-read-parquet-file-that-is-using-zstd-codec/32657)

<div class="topic-metadata">

**Author:** [@ldsands](https://discourse.julialang.org/u/ldsands)\
**Replies:** 6\
**Last updated:** [May 6, 2020, 5:45am UTC](https://discourse.julialang.org/t/trying-to-read-parquet-file-that-is-using-zstd-codec/32657 "2020-05-06T05:45:26Z")

</div>

Hello everyone, I would like to read a parquet file into a Julia DataFrame. Unfortunately for my Julia skills (but fortunately for my hard drive) the codec used to compress the parquet file was zstd. I have played aroun…