# Reading Parquet file into Apache Arrow?

**URL:** https://discourse.julialang.org/t/reading-parquet-file-into-apache-arrow/50851
**Category:** Data
**Tags:** dataframes
**Created:** [November 27, 2020, 8:37am UTC](https://discourse.julialang.org/t/reading-parquet-file-into-apache-arrow/50851 "2020-11-27T08:37:43Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![s-kap](https://avatars.discourse-cdn.com/v4/letter/s/ba8739/32.png) [@s-kap](https://discourse.julialang.org/u/s-kap)
#### Post date: [November 27, 2020, 8:37am UTC](https://discourse.julialang.org/t/reading-parquet-file-into-apache-arrow/50851/1 "2020-11-27T08:37:44Z")

</div>

Hey hey, what’s the easiest way to read a parquet file into an apache arrow dataset? In python one can simply do `pyarrow.parquet.read_table(..)`. I know Parquet.jl and Arrow.jl exist, but I haven’t found a way to make the two work together.

My goal is to read the parquet file into a dataframe and perform some benchmarks comparing regular DataFrames vs DataFrames using Apache Arrow vecs.

The only way I see to convert Parquet files to a DataFrame is by using the RecordCursor, which doesn’t seem ideal because parquet is a columnar format and so are DataFrames and Apache Arrow - and having to iterate over rows is really slow

---

<div class="post-metadata">

### Author: ![nalimilan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nalimilan/32/147_2.png) [@nalimilan](https://discourse.julialang.org/u/nalimilan)
#### Post date: [November 27, 2020, 8:47am UTC](https://discourse.julialang.org/t/reading-parquet-file-into-apache-arrow/50851/2 "2020-11-27T08:47:14Z")

</div>

I think you can use ParquetFiles.jl.

---

<div class="post-metadata">

### Author: ![Rudi79](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rudi79/32/3884_2.png) [@Rudi79](https://discourse.julialang.org/u/Rudi79)
#### Post date: [November 27, 2020, 9:00am UTC](https://discourse.julialang.org/t/reading-parquet-file-into-apache-arrow/50851/3 "2020-11-27T09:00:35Z")

</div>

I used [https://github.com/lungben/TableIO.jl](https://github.com/lungben/TableIO.jl) for a similar task.

---

<div class="post-metadata">

### Author: ![lungben](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lungben/32/12314_2.png) [@lungben](https://discourse.julialang.org/u/lungben)
#### Post date: [November 27, 2020, 10:15am UTC](https://discourse.julialang.org/t/reading-parquet-file-into-apache-arrow/50851/4 "2020-11-27T10:15:36Z")

</div>

TableIO uses Parquet.jl internally for this task.

---

<div class="post-metadata">

### Author: ![s-kap](https://avatars.discourse-cdn.com/v4/letter/s/ba8739/32.png) [@s-kap](https://discourse.julialang.org/u/s-kap)
#### Post date: [November 27, 2020, 11:16am UTC](https://discourse.julialang.org/t/reading-parquet-file-into-apache-arrow/50851/5 "2020-11-27T11:16:21Z")

</div>

ParquetFiles was last updated about 2 years ago and doesn’t work anymore ☹

---

<div class="post-metadata">

### Author: ![s-kap](https://avatars.discourse-cdn.com/v4/letter/s/ba8739/32.png) [@s-kap](https://discourse.julialang.org/u/s-kap)
#### Post date: [November 27, 2020, 11:17am UTC](https://discourse.julialang.org/t/reading-parquet-file-into-apache-arrow/50851/6 "2020-11-27T11:17:18Z")

</div>

Thanks!  
This looks great!