# Is it possible to join DataFrame with Arrow Table ensuring unique rows without bringing Arrow Table into RAM?

**URL:** https://discourse.julialang.org/t/is-it-possible-to-join-dataframe-with-arrow-table-ensuring-unique-rows-without-bringing-arrow-table-into-ram/96873
**Category:** New to Julia
**Tags:** question, dataframes, arrow
**Created:** [March 31, 2023, 4:53am UTC](https://discourse.julialang.org/t/is-it-possible-to-join-dataframe-with-arrow-table-ensuring-unique-rows-without-bringing-arrow-table-into-ram/96873 "2023-03-31T04:53:04Z")
**Posts on this page:** 1
**Showing post:** 4

<div class="post-metadata">

### Author: ![phantom](https://avatars.discourse-cdn.com/v4/letter/p/e0b2c6/32.png) [@phantom](https://discourse.julialang.org/u/phantom)
#### Post date: [April 3, 2023, 11:37pm UTC](https://discourse.julialang.org/t/is-it-possible-to-join-dataframe-with-arrow-table-ensuring-unique-rows-without-bringing-arrow-table-into-ram/96873/4 "2023-04-03T23:37:44Z")

</div>

So combining @JorizovdZ and @rocco_sprmnt21’s solution [here](https://discourse.julialang.org/t/delete-row-from-dataframe-in-place-based-on-entire-row-value/96961/4) and @bkamins solution [here](https://discourse.julialang.org/t/arrow-stream-usage-clarification/89508/4) thus far I think the best way to combine a new `DataFrame` with an existing `Arrow` file ensuring unique values without bringing the `Arrow` file into memory would be something like

```julia
database = "filepath" 
Arrow.write(database ,dataDF, file = false) # when saving the dataDF as arrow file - keyword file has to be set to false 
dataDF = DataFrame(Arrow.Table(database))   
NewDF = antijoin(NewDF, dataDF , on = intersect(names(NewDF), names(DataDF)) 
Arrow.append (database, NewDF)

```

---

_[View the full topic](https://discourse.julialang.org/t/is-it-possible-to-join-dataframe-with-arrow-table-ensuring-unique-rows-without-bringing-arrow-table-into-ram/96873)._