# Tables partitions

**URL:** https://discourse.julialang.org/t/tables-partitions/95930
**Category:** New to Julia
**Tags:** tables
**Created:** [March 11, 2023, 5:52pm UTC](https://discourse.julialang.org/t/tables-partitions/95930 "2023-03-11T17:52:42Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![freeman](https://avatars.discourse-cdn.com/v4/letter/f/ec9cab/32.png) [@freeman](https://discourse.julialang.org/u/freeman)
#### Post date: [March 11, 2023, 5:52pm UTC](https://discourse.julialang.org/t/tables-partitions/95930/1 "2023-03-11T17:52:42Z")

</div>

Regarding the Tables.jl interface, I find the part about partitions quite confusing. For starters it’s quite sparse, as far as I can tell there’s only a few paragraphs [1] on `Tables.partitions` and `Tables.partitioner`. And then it doesn’t actually give you any context. It doesn’t tell you what problem is being solved, it just tells you what actions are performed by calling the functions.

I have the following specific problem, if anyone knows the answer:

Conceptually, lets say I have a partitioned table:

```julia
using DataFrames
df1 = DataFrame(:a=>[1,2,3,4,5,6,7])
df2 = DataFrame(:a=>[8,9,10])
t = [df1, df2]

```

I call this a partitioned table because I can iterate over it, and each element is a `Tables.istable`. However, `t` by itself is not `Tables.istable`. _So if I want to make `t` into something conforming to the Tables.jl interface, what do I have to do?_ Something that I can call `Tables.rows` on, and will iterate first over `df1` then over `df2` seamlessly.

At first I thought this is the point of either `Tables.partitions` or `Tables.partitioner` - take an iterator of Tables and return a Table. But no, that’s not the case because neither of the things below work:

```julia
t |> Tables.partitions |> Tables.istable # false
t |> Tables.partitioner |> Tables.istable # false

```

[1] [Home · Tables.jl](https://tables.juliadata.org/stable/#Tables.partitions)

---

<div class="post-metadata">

### Author: ![jd-foster](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jd-foster/32/35824_2.png) [@jd-foster](https://discourse.julialang.org/u/jd-foster)
#### Post date: [March 11, 2023, 9:37pm UTC](https://discourse.julialang.org/t/tables-partitions/95930/2 "2023-03-11T21:37:16Z")

</div>

You might find this video from @quinnj useful:

[![](https://global.discourse-cdn.com/julialang/original/3X/3/d/3de2b8ef32b8b78a2b7734d6cc9abfc944527af8.jpeg "Partitions and chains: enabling batch processing for your data | Jacob Quinn | JuliaCon2021") ](https://www.youtube.com/watch?v=Ryp0F0dHnLA)

---

<div class="post-metadata">

### Author: ![bkamins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bkamins/32/208538_2.png) [@bkamins](https://discourse.julialang.org/u/bkamins)
#### Post date: [March 11, 2023, 9:48pm UTC](https://discourse.julialang.org/t/tables-partitions/95930/3 "2023-03-11T21:48:28Z")

</div>

I assume you want this:

```julia
julia> for row in Tables.rows(TableOperations.joinpartitions(Tables.partitioner([df1, df2])))
           @show row
       end
row = Tables.ColumnsRow{Tables.CopiedColumns{TableOperations.JoinedPartitions{Tables.Schema{(:a,), Tuple{Int64}}}}}:
 :a 1
row = Tables.ColumnsRow{Tables.CopiedColumns{TableOperations.JoinedPartitions{Tables.Schema{(:a,), Tuple{Int64}}}}}:
 :a 2
row = Tables.ColumnsRow{Tables.CopiedColumns{TableOperations.JoinedPartitions{Tables.Schema{(:a,), Tuple{Int64}}}}}:
 :a 3
row = Tables.ColumnsRow{Tables.CopiedColumns{TableOperations.JoinedPartitions{Tables.Schema{(:a,), Tuple{Int64}}}}}:
 :a 4
row = Tables.ColumnsRow{Tables.CopiedColumns{TableOperations.JoinedPartitions{Tables.Schema{(:a,), Tuple{Int64}}}}}:
 :a 5
row = Tables.ColumnsRow{Tables.CopiedColumns{TableOperations.JoinedPartitions{Tables.Schema{(:a,), Tuple{Int64}}}}}:
 :a 6
row = Tables.ColumnsRow{Tables.CopiedColumns{TableOperations.JoinedPartitions{Tables.Schema{(:a,), Tuple{Int64}}}}}:
 :a 7
row = Tables.ColumnsRow{Tables.CopiedColumns{TableOperations.JoinedPartitions{Tables.Schema{(:a,), Tuple{Int64}}}}}:
 :a 8
row = Tables.ColumnsRow{Tables.CopiedColumns{TableOperations.JoinedPartitions{Tables.Schema{(:a,), Tuple{Int64}}}}}:
 :a 9
row = Tables.ColumnsRow{Tables.CopiedColumns{TableOperations.JoinedPartitions{Tables.Schema{(:a,), Tuple{Int64}}}}}:
 :a 10

```

---

<div class="post-metadata">

### Author: ![freeman](https://avatars.discourse-cdn.com/v4/letter/f/ec9cab/32.png) [@freeman](https://discourse.julialang.org/u/freeman)
#### Post date: [March 11, 2023, 10:20pm UTC](https://discourse.julialang.org/t/tables-partitions/95930/4 "2023-03-11T22:20:59Z")

</div>

> [@bkamins](#):
>
> `TableOperations`

Fantastic, thank you.

---

<div class="post-metadata">

### Author: ![rocco\_sprmnt21](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rocco_sprmnt21/32/20127_2.png) [@rocco\_sprmnt21](https://discourse.julialang.org/u/rocco_sprmnt21)
#### Post date: [March 12, 2023, 5:21pm UTC](https://discourse.julialang.org/t/tables-partitions/95930/5 "2023-03-12T17:21:34Z")

</div>

and then, of course,

```julia
julia> Tables.istable(TableOperations.joinpartitions(Tables.partitioner(t)))
true

```
