# \#arrow

**URL:** https://discourse.julialang.org/tag/arrow/808.md

[Latest](https://discourse.julialang.org/latest.md) · [Categories](https://discourse.julialang.org/categories.md) · [Tags](https://discourse.julialang.org/tags.md)

---

## [\[RFC/ANN\] Parquet3.jl — alternative Parquet implementation focused on support for nested columns](https://discourse.julialang.org/t/rfc-ann-parquet3-jl-alternative-parquet-implementation-focused-on-support-for-nested-columns/136295)

<div class="topic-metadata">

**Author:** [@Yuan-Ru-Lin](https://discourse.julialang.org/u/Yuan-Ru-Lin)\
**Replies:** 9\
**Last updated:** [April 10, 2026, 2:59am UTC](https://discourse.julialang.org/t/rfc-ann-parquet3-jl-alternative-parquet-implementation-focused-on-support-for-nested-columns/136295 "2026-04-10T02:59:17Z")

</div>

I’d like to announce and request comments on Parquet3.jl. This implementation focuses specifically on support for nested column types, which are explicitly out of scope for Parquet2.jl, but would be a suitable representa…

---

## [Does Arrow.jl support enums?](https://discourse.julialang.org/t/does-arrow-jl-support-enums/89163)

<div class="topic-metadata">

**Author:** [@freeman](https://discourse.julialang.org/u/freeman)\
**Replies:** 13\
**Last updated:** [March 5, 2026, 8:23pm UTC](https://discourse.julialang.org/t/does-arrow-jl-support-enums/89163 "2026-03-05T20:23:01Z")

</div>

I’m surprised that the code below doesn’t work. @enum MyEnum LALA1 LALA2 LALA3 x = \[LALA1, LALA1, LALA3, LALA2\] table = (cols1=x,) io = IOBuffer() Arrow.write(io, table) ERROR: MethodError: no method matching arrowtype…

---

## [Makie: plotting the sample arrow for \`arrows!()\`?](https://discourse.julialang.org/t/makie-plotting-the-sample-arrow-for-arrows/106642)

<div class="topic-metadata">

**Author:** [@ryofurue](https://discourse.julialang.org/u/ryofurue)\
**Replies:** 1\
**Last updated:** [January 23, 2026, 1:02pm UTC](https://discourse.julialang.org/t/makie-plotting-the-sample-arrow-for-arrows/106642 "2026-01-23T13:02:37Z")

</div>

I’ve just started to use arrows!(). I’ve find it straightforward to use. Now, I’m trying to find a way to plot the sample arrow. See the image, which I plotted using a different program, at the bottom of this message. …

---

## [Arrows for annotation in Makie.jl](https://discourse.julialang.org/t/arrows-for-annotation-in-makie-jl/108374)

<div class="topic-metadata">

**Author:** [@nari](https://discourse.julialang.org/u/nari)\
**Replies:** 4\
**Last updated:** [April 29, 2025, 4:37pm UTC](https://discourse.julialang.org/t/arrows-for-annotation-in-makie-jl/108374 "2025-04-29T16:37:30Z")

</div>

Makie.jl has arrows, which are suitable for illustrating vector fields, and annotations seem to be out of their intended use. If we want to annotate with arrows like matplotlib’s anotations, what is the best way in Maki…

---

## [Type conversion](https://discourse.julialang.org/t/type-conversion/128012)

<div class="topic-metadata">

**Author:** [@ufechner7](https://discourse.julialang.org/u/ufechner7)\
**Replies:** 1\
**Last updated:** [April 13, 2025, 7:45pm UTC](https://discourse.julialang.org/t/type-conversion/128012 "2025-04-13T19:45:50Z")

</div>

After reading an arrow file I get the following type: julia\> sl.heading 200-element Arrow.Primitive{Float32, Vector{Float32}}: -0.0023596277 -0.004771689 -0.0052604624 ⋮ -0.31095406 -0.33066812 -0.338615 If I w…

---

## [Arrow's DictEncode to CategoricalArray?](https://discourse.julialang.org/t/arrows-dictencode-to-categoricalarray/112451)

<div class="topic-metadata">

**Author:** [@elenev](https://discourse.julialang.org/u/elenev)\
**Replies:** 18\
**Last updated:** [February 11, 2025, 2:30pm UTC](https://discourse.julialang.org/t/arrows-dictencode-to-categoricalarray/112451 "2025-02-11T14:30:28Z")

</div>

I am using Arrow.jl to store a dataframe to disk. The dataframe has some columns of CategoricalArray{T} type, where T is either String or Int64. When I read the dataframe back in, the categorical-ness of these columns i…

---

## [Why JLD2.jl is 40x slower than Arrow.jl](https://discourse.julialang.org/t/why-jld2-jl-is-40x-slower-than-arrow-jl/122217)

<div class="topic-metadata">

**Author:** [@Sixzero](https://discourse.julialang.org/u/Sixzero)\
**Replies:** 24\
**Last updated:** [November 25, 2024, 9:59am UTC](https://discourse.julialang.org/t/why-jld2-jl-is-40x-slower-than-arrow-jl/122217 "2024-11-25T09:59:38Z")

</div>

I have this gist: JLD2 is actually simply for reading in a 30 MB file with keys =\> randn(500) is 40x slower than Arrow.jl. How? I mean how come JLD2 generality sacrificed 40x speed difference?

---

## [Optimize Apache Arrow data streaming over HTTP](https://discourse.julialang.org/t/optimize-apache-arrow-data-streaming-over-http/113692)

<div class="topic-metadata">

**Author:** [@simsurace](https://discourse.julialang.org/u/simsurace)\
**Replies:** 29\
**Last updated:** [October 12, 2024, 6:56pm UTC](https://discourse.julialang.org/t/optimize-apache-arrow-data-streaming-over-http/113692 "2024-10-12T18:56:55Z")

</div>

Following a request on the Arrow.jl project, I started implementing a simple Julia example here. With the exeption of an upstream issue on macOS, it seems to work. However, the implementation seems to be less performant …

---

## [Failing to import (relatively) large CSV file with Julia and VSC](https://discourse.julialang.org/t/failing-to-import-relatively-large-csv-file-with-julia-and-vsc/112350)

<div class="topic-metadata">

**Author:** [@etiennebert](https://discourse.julialang.org/u/etiennebert)\
**Replies:** 24\
**Last updated:** [September 22, 2024, 12:53pm UTC](https://discourse.julialang.org/t/failing-to-import-relatively-large-csv-file-with-julia-and-vsc/112350 "2024-09-22T12:53:54Z")

</div>

Hello everyone, I am completly new to Julia, so please my apologies if my question is trivial. I have quickly looked for the reply and I have not found it. I need to import a large matrix (40 000 x 40 000, app 6GB) int…

---

## [Problem with numerical data in Arrow.jl](https://discourse.julialang.org/t/problem-with-numerical-data-in-arrow-jl/117218)

<div class="topic-metadata">

**Author:** [@Giobbe](https://discourse.julialang.org/u/Giobbe)\
**Replies:** 1\
**Last updated:** [July 19, 2024, 8:58am UTC](https://discourse.julialang.org/t/problem-with-numerical-data-in-arrow-jl/117218 "2024-07-19T08:58:56Z")

</div>

Hello to everyBody. I am trying to use Arrow.jl to store my DataFrames from a e-commerce DB and I noticed that the import function doesnt convert properly decimal values in the dataframes. I post the comparison with CS…

---

## [Why does Appending a Dataframe to an Arrow file Change the column type?](https://discourse.julialang.org/t/why-does-appending-a-dataframe-to-an-arrow-file-change-the-column-type/115378)

<div class="topic-metadata">

**Author:** [@phantom](https://discourse.julialang.org/u/phantom)\
**Replies:** 1\
**Last updated:** [June 9, 2024, 7:37am UTC](https://discourse.julialang.org/t/why-does-appending-a-dataframe-to-an-arrow-file-change-the-column-type/115378 "2024-06-09T07:37:32Z")

</div>

Hi, Suppose I have the following DataFrame with non specified Column type. Q = DataFrame(datmonth = \[\], datstat =\[\], qrytime = \[\]) 0×3 DataFrame Row │ datmonth datstat qrytime │ Any Any Any ───…

---

## [Fastest way to save a large number of DataFrames to disk](https://discourse.julialang.org/t/fastest-way-to-save-a-large-number-of-dataframes-to-disk/114005)

<div class="topic-metadata">

**Author:** [@liuyxpp](https://discourse.julialang.org/u/liuyxpp)\
**Replies:** 2\
**Last updated:** [May 10, 2024, 12:07pm UTC](https://discourse.julialang.org/t/fastest-way-to-save-a-large-number-of-dataframes-to-disk/114005 "2024-05-10T12:07:51Z")

</div>

I have a large number of dataframes generated from other sources like using DataFrames N = 1000 df\_dict = Dict(1:N .=\> \[DataFrame(rand(rand(1000:2000), 146), :auto) for \_ in 1:N\]) I’d like to save them into a single fi…

---

## [Outputting string columns using Avro.jl](https://discourse.julialang.org/t/outputting-string-columns-using-avro-jl/80112)

<div class="topic-metadata">

**Author:** [@kobusherbst](https://discourse.julialang.org/u/kobusherbst)\
**Replies:** 5\
**Last updated:** [April 15, 2024, 5:13pm UTC](https://discourse.julialang.org/t/outputting-string-columns-using-avro-jl/80112 "2024-04-15T17:13:51Z")

</div>

When I try to convert an Arrow file to Avro using this code: function t\_arrowtoavro(path, file) a = Arrow.Table(joinpath(path, "$(file).arrow")) |\> DataFrame println(Tables.schema(a)) Avro.writetable(joinpat…

---

## [How to release lock on Arrow table](https://discourse.julialang.org/t/how-to-release-lock-on-arrow-table/107046)

<div class="topic-metadata">

**Author:** [@taotree](https://discourse.julialang.org/u/taotree)\
**Replies:** 7\
**Last updated:** [December 8, 2023, 2:23pm UTC](https://discourse.julialang.org/t/how-to-release-lock-on-arrow-table/107046 "2023-12-08T14:23:01Z")

</div>

I need to read in an Arrow file, combine some data, and then overwrite that file. That means I need to release the file lock that happens when you read. How can I guarantee to release that lock? I’ve read that clearing o…

---

## [Why trying to remove .arrow file fails with IOError on Windows?](https://discourse.julialang.org/t/why-trying-to-remove-arrow-file-fails-with-ioerror-on-windows/105273)

<div class="topic-metadata">

**Author:** [@Tortar](https://discourse.julialang.org/u/Tortar)\
**Replies:** 2\
**Last updated:** [October 23, 2023, 4:56pm UTC](https://discourse.julialang.org/t/why-trying-to-remove-arrow-file-fails-with-ioerror-on-windows/105273 "2023-10-23T16:56:31Z")

</div>

This is (hopefully) an MWE of my actual problem: using Arrow, DataFrames if true data = DataFrame(a=1:4, b=\["M", "F", "F", "M"\]) Arrow.write("MWE.arrow", data; file = false) Arrow.Stream("MWE.arrow") GC.…

---

## [Releasing process memory from Arrow.jl](https://discourse.julialang.org/t/releasing-process-memory-from-arrow-jl/102882)

<div class="topic-metadata">

**Author:** [@djholiver](https://discourse.julialang.org/u/djholiver)\
**Replies:** 5\
**Last updated:** [August 17, 2023, 7:23pm UTC](https://discourse.julialang.org/t/releasing-process-memory-from-arrow-jl/102882 "2023-08-17T19:23:40Z")

</div>

Hi, I make heavy use of the (incredible) Arrow.jl package provided by @quinnj to do api - exposed analytics running in docker containers. The files are large: c. 65GB, 35M rows, 250 columns and the analytics queries exe…

---

## [Arrow stream usage clarification](https://discourse.julialang.org/t/arrow-stream-usage-clarification/89508)

<div class="topic-metadata">

**Author:** [@jeremiedb](https://discourse.julialang.org/u/jeremiedb)\
**Replies:** 10\
**Last updated:** [July 17, 2023, 7:55pm UTC](https://discourse.julialang.org/t/arrow-stream-usage-clarification/89508 "2023-07-17T19:55:57Z")

</div>

I’m trying to use Arrow’s Stream https://arrow.juliadata.org/dev/reference/#Arrow.Stream to get an efficient out of RAM tabular data iterator. However, attempts to iterate over the arrow stream only appear to perform a …

---

## [Arrow stream writer and reader implementation questions](https://discourse.julialang.org/t/arrow-stream-writer-and-reader-implementation-questions/101438)

<div class="topic-metadata">

**Author:** [@schaefer0](https://discourse.julialang.org/u/schaefer0)\
**Replies:** 0\
**Last updated:** [July 10, 2023, 2:59pm UTC](https://discourse.julialang.org/t/arrow-stream-writer-and-reader-implementation-questions/101438 "2023-07-10T14:59:43Z")

</div>

Hi, I was using the python arrow function record\_batch and find that this does not exist in julia, but there is an arrow table function. However they don’t behave the same way. I have a customer requirement (that I can…

---

## [Makie - arrowhead options](https://discourse.julialang.org/t/makie-arrowhead-options/100596)

<div class="topic-metadata">

**Author:** [@yoplo](https://discourse.julialang.org/u/yoplo)\
**Replies:** 4\
**Last updated:** [June 20, 2023, 5:14pm UTC](https://discourse.julialang.org/t/makie-arrowhead-options/100596 "2023-06-20T17:14:19Z")

</div>

Hello! I am drawing an arrows plot in GLMakie, Where can I find a list of the legal inputs to the arrowhead keyword? yoplo

---

## [Pyarrow conversion with PythonCall](https://discourse.julialang.org/t/pyarrow-conversion-with-pythoncall/99485)

<div class="topic-metadata">

**Author:** [@tbeason](https://discourse.julialang.org/u/tbeason)\
**Replies:** 11\
**Last updated:** [May 27, 2023, 4:03pm UTC](https://discourse.julialang.org/t/pyarrow-conversion-with-pythoncall/99485 "2023-05-27T16:03:17Z")

</div>

It would appear to me that PythonCall.jl does not convert pyarrow tables to DataFrames. I get the error ERROR: cannot convert this Python 'Table' to a Julia 'PyTable' Since PyTable(x) wraps Tables.jl compatible tables,…

---

## [Arrow changes a DataFrame column from type \`Float32\` to \`Float32?\` without missing values?](https://discourse.julialang.org/t/arrow-changes-a-dataframe-column-from-type-float32-to-float32-without-missing-values/99045)

<div class="topic-metadata">

**Author:** [@phantom](https://discourse.julialang.org/u/phantom)\
**Replies:** 5\
**Last updated:** [May 22, 2023, 2:03pm UTC](https://discourse.julialang.org/t/arrow-changes-a-dataframe-column-from-type-float32-to-float32-without-missing-values/99045 "2023-05-22T14:03:44Z")

</div>

I’m sure I’m overlooking something simple here but is there a reason why Arrow would convert a Float32 column in a Dataframe df to type Float32? when all(x-\>typeof(x) == Float32, df.col) true and any(ismissing.(df.col…

---

## [Cannot read file written by Arrow.jl in Python](https://discourse.julialang.org/t/cannot-read-file-written-by-arrow-jl-in-python/97889)

<div class="topic-metadata">

**Author:** [@heliosdrm](https://discourse.julialang.org/u/heliosdrm)\
**Replies:** 3\
**Last updated:** [April 25, 2023, 12:36pm UTC](https://discourse.julialang.org/t/cannot-read-file-written-by-arrow-jl-in-python/97889 "2023-04-25T12:36:55Z")

</div>

I’m using a Julia environment with Arrow v2.5.2 and DataFrames v1.5.1, and Python 3.10.2 with pandas 1.5.0 and pyarrow 11.0.0. From the User manual I think I could do: using Arrow, DataFrames m = rand(20,10) open("exa…

---

## [Append a \`DataFrame\` to a partition of an existing \`ArrowTable\` without creating a new \`ArrowTable\`?](https://discourse.julialang.org/t/append-a-dataframe-to-a-partition-of-an-existing-arrowtable-without-creating-a-new-arrowtable/97461)

<div class="topic-metadata">

**Author:** [@phantom](https://discourse.julialang.org/u/phantom)\
**Replies:** 7\
**Last updated:** [April 21, 2023, 1:28am UTC](https://discourse.julialang.org/t/append-a-dataframe-to-a-partition-of-an-existing-arrowtable-without-creating-a-new-arrowtable/97461 "2023-04-21T01:28:19Z")

</div>

Hi! suppose I have the following GroupedDataFrame GDF1 = groupby(DataFrame(ID = \[ "Eng1", "Eng2"\] , Date = \[Date(2023,4,10),Date(2023,4,10)\], Time = \[3.85, 4.13\]), :ID) With Arrow.append I can save each subdataframe o…

---

## [Write data to Arrow file row by row](https://discourse.julialang.org/t/write-data-to-arrow-file-row-by-row/59165)

<div class="topic-metadata">

**Author:** [@danielw2904](https://discourse.julialang.org/u/danielw2904)\
**Replies:** 7\
**Last updated:** [April 7, 2023, 9:31pm UTC](https://discourse.julialang.org/t/write-data-to-arrow-file-row-by-row/59165 "2023-04-07T21:31:06Z")

</div>

I have a large amount of data I am loading from an API and I would like to write the output of each call to a “row” of a file in order not to keep it in RAM/ in case the program errors. I have been using the Arrow.jl pac…

---

## [Is it possible to join DataFrame with Arrow Table ensuring unique rows without bringing Arrow Table into RAM?](https://discourse.julialang.org/t/is-it-possible-to-join-dataframe-with-arrow-table-ensuring-unique-rows-without-bringing-arrow-table-into-ram/96873)

<div class="topic-metadata">

**Author:** [@phantom](https://discourse.julialang.org/u/phantom)\
**Replies:** 3\
**Last updated:** [April 3, 2023, 11:37pm UTC](https://discourse.julialang.org/t/is-it-possible-to-join-dataframe-with-arrow-table-ensuring-unique-rows-without-bringing-arrow-table-into-ram/96873 "2023-04-03T23:37:44Z")

</div>

I’m sure there is probably a better approach I am not aware of but I am trying to add a new DataFrame, NewDF to an existing Arrow table while eliminating any rows in NewDF that already exist in the Arrow file. Somethin…

---

## [Further Clarification on scoping rules](https://discourse.julialang.org/t/further-clarification-on-scoping-rules/96373)

<div class="topic-metadata">

**Author:** [@phantom](https://discourse.julialang.org/u/phantom)\
**Replies:** 3\
**Last updated:** [March 21, 2023, 7:21am UTC](https://discourse.julialang.org/t/further-clarification-on-scoping-rules/96373 "2023-03-21T07:21:37Z")

</div>

Sorry if I am overlooking something again. But according to the documentation on Scope: Variables in outer scopes are visible from any scope they contain — meaning that they can be read and written in inner scopes — un…

---

## [Storing huge amount of data efficiently](https://discourse.julialang.org/t/storing-huge-amount-of-data-efficiently/94818)

<div class="topic-metadata">

**Author:** [@Dario-Rosa85](https://discourse.julialang.org/u/Dario-Rosa85)\
**Replies:** 15\
**Last updated:** [February 24, 2023, 9:21am UTC](https://discourse.julialang.org/t/storing-huge-amount-of-data-efficiently/94818 "2023-02-24T09:21:37Z")

</div>

Hello, I am performing some numerical computations and I have the necessity to run a huge (say, several thousand of iterations) for loop. Each iteration produces a certain amount of data that I need to store in the disk…

---

## [Setindex! issue with DataFrame saved as Arrow file](https://discourse.julialang.org/t/setindex-issue-with-dataframe-saved-as-arrow-file/90307)

<div class="topic-metadata">

**Author:** [@alex-s-gardner](https://discourse.julialang.org/u/alex-s-gardner)\
**Replies:** 9\
**Last updated:** [November 16, 2022, 1:03am UTC](https://discourse.julialang.org/t/setindex-issue-with-dataframe-saved-as-arrow-file/90307 "2022-11-16T01:03:23Z")

</div>

Working with a DataFrame saved as an Arrow file: fn = download("https://www.dropbox.com/s/322g26p3apdeqpf/granules.remote?dl=0") Loading in the file as a df g = DataFrame(Arrow.Table(fn)) Copy the DataFrame g0 = cop…

---

## [Help with Arrow.jl and size of files](https://discourse.julialang.org/t/help-with-arrow-jl-and-size-of-files/89028)

<div class="topic-metadata">

**Author:** [@austin-putz](https://discourse.julialang.org/u/austin-putz)\
**Replies:** 23\
**Last updated:** [October 21, 2022, 2:40am UTC](https://discourse.julialang.org/t/help-with-arrow-jl-and-size-of-files/89028 "2022-10-21T02:40:20Z")

</div>

I was recently exploring the Arrow.jl package and arrow package in R. I had a project where I needed to write a function to convert a large file in Julia, then port it to R for another package we need to use there. I wr…

---

## [Senior High-Performance Software Engineer - Liquid Analytics](https://discourse.julialang.org/t/senior-high-performance-software-engineer-liquid-analytics/84689)

<div class="topic-metadata">

**Author:** [@Nathalie\_Burstein](https://discourse.julialang.org/u/Nathalie_Burstein)\
**Replies:** 0\
**Last updated:** [July 23, 2022, 11:09pm UTC](https://discourse.julialang.org/t/senior-high-performance-software-engineer-liquid-analytics/84689 "2022-07-23T23:09:02Z")

</div>

We solve unique, complex problems with highly dynamic data sets. Be part of a team building a specific high-performance mainline database with Apache Arrow. Get the best of both worlds! Liquid Analytics is a vibrant team…

[Next page](https://discourse.julialang.org/tag/arrow/808.md?match_all_tags=true&page=1&tags%5B%5D=arrow)