# How to properly add two new columns (in dataframe) from a function that returns two arrays?

**URL:** <https://discourse.julialang.org/t/how-to-properly-add-two-new-columns-in-dataframe-from-a-function-that-returns-two-arrays/63964>\
**Category:** General Usage\
**Tags:** dataframes, function\
**Created:** [July 2, 2021, 11:48pm UTC](https://discourse.julialang.org/t/how-to-properly-add-two-new-columns-in-dataframe-from-a-function-that-returns-two-arrays/63964 "2021-07-02T23:48:33Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![joonp](https://avatars.discourse-cdn.com/v4/letter/j/a9a28c/32.png) [@joonp](https://discourse.julialang.org/u/joonp)\
**Post date:** [July 2, 2021, 11:48pm UTC](https://discourse.julialang.org/t/how-to-properly-add-two-new-columns-in-dataframe-from-a-function-that-returns-two-arrays/63964/1 "2021-07-02T23:48:33Z")

</div>

Hello, I am a Julia newbie and I have a question.  
The example below sort of simplified my current problem—

```julia
function myfun(id)
  a = [id .+ 1, id .+ 2, id .+ 2]
  b = [id .- 1, id .- 2, id .- 3]
  return [a, b]
end

mydf = DataFrame(id = [7, 8])
combine(groupby(mydf, :id), :id => myfun)

```

For each group (groupby `id` in my data `mydf`), I run a function, `myfun`, which generates two arrays based on a variable in `mydf`.  
What I want to do is to add the two arrays (`a` and `b`) from the function to the original data as two columns.

However… the command below generates only a single column

```julia
combine(groupby(mydf, :id), :id => myfun)

4×2 DataFrame
 Row │ id id_myfun          
     │ Int64 Array…            
─────┼──────────────────────────
   1 │ 7 [[8], [9], [9]]
   2 │ 7 [[6], [5], [4]]
   3 │ 8 [[9], [10], [10]]
   4 │ 8 [[7], [6], [5]]

```

Ideally, I want my result to be like below…

```julia
 Row │ id │ a │b 
     │ Int64 │ Int64 │ Int64
─────┼──────────────────────────
   1 │ 7 | 8 │ 6 
   2 │ 7 | 9 │ 5
   3 │ 7 | 9 │ 4 
   4 │ 7 | 9 │ 7 
   5 │ 7 | 10 │ 6
   6 │ 7 | 10 │ 5 

```

Is there anyone who could help me?

Thanks!

---

<div class="post-metadata">

**Author:** ![sijo](https://avatars.discourse-cdn.com/v4/letter/s/da6949/32.png) [@sijo](https://discourse.julialang.org/u/sijo)\
**Post date:** [July 3, 2021, 6:34am UTC](https://discourse.julialang.org/t/how-to-properly-add-two-new-columns-in-dataframe-from-a-function-that-returns-two-arrays/63964/2 "2021-07-03T06:34:55Z")

</div>

Here’s one way:

```julia
julia> df_ab = transform(mydf, :id => ByRow(myfun) => [:a, :b])
2×3 DataFrame
 Row │ id a b         
     │ Int64 Array… Array…    
─────┼───────────────────────────────
   1 │ 7 [8, 9, 9] [6, 5, 4]
   2 │ 8 [9, 10, 10] [7, 6, 5]

julia> flatten(df_ab, [:a, :b])
6×3 DataFrame
 Row │ id a b     
     │ Int64 Int64 Int64 
─────┼─────────────────────
   1 │ 7 8 6
   2 │ 7 9 5
   3 │ 7 9 4
   4 │ 8 9 7
   5 │ 8 10 6
   6 │ 8 10 5

```

---

<div class="post-metadata">

**Author:** ![rikh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rikh/32/204104_2.png) [@rikh](https://discourse.julialang.org/u/rikh)\
**Post date:** [July 3, 2021, 10:58am UTC](https://discourse.julialang.org/t/how-to-properly-add-two-new-columns-in-dataframe-from-a-function-that-returns-two-arrays/63964/4 "2021-07-03T10:58:00Z")

</div>

Another way

```julia
julia> function myfun(id)
           a = [id .+ 1, id .+ 2, id .+ 2]
           b = [id .- 1, id .- 2, id .- 3]
           id = fill(id, length(a))
           (; id, a, b)
       end;

julia> vcat(DataFrame.(myfun.([7, 8]))...)
6×3 DataFrame
 Row │ id a b
     │ Int64 Int64 Int64
─────┼─────────────────────
   1 │ 7 8 6
   2 │ 7 9 5
   3 │ 7 9 4
   4 │ 8 9 7
   5 │ 8 10 6
   6 │ 8 10 5

```

---

<div class="post-metadata">

**Author:** ![joonp](https://avatars.discourse-cdn.com/v4/letter/j/a9a28c/32.png) [@joonp](https://discourse.julialang.org/u/joonp)\
**Post date:** [July 3, 2021, 12:46pm UTC](https://discourse.julialang.org/t/how-to-properly-add-two-new-columns-in-dataframe-from-a-function-that-returns-two-arrays/63964/5 "2021-07-03T12:46:53Z")

</div>

Many Thanks @sijo and @rikh!  
These are perfect answers for me.

---

<div class="post-metadata">

**Author:** ![sijo](https://avatars.discourse-cdn.com/v4/letter/s/da6949/32.png) [@sijo](https://discourse.julialang.org/u/sijo)\
**Post date:** [July 3, 2021, 1:01pm UTC](https://discourse.julialang.org/t/how-to-properly-add-two-new-columns-in-dataframe-from-a-function-that-returns-two-arrays/63964/6 "2021-07-03T13:01:31Z")

</div>

Some alternatives for reference 🙂

```julia
combine(groupby(mydf, :id), :id => Base.splat(hcat)∘myfun∘only => [:a, :b])

combine(groupby(mydf, :id), :id => NamedTuple{(:a, :b)}∘myfun∘only => AsTable)

# If the `id` column is not required in the result:
combine(mydf, :id => Base.splat(vcat)∘ByRow(Base.splat(hcat)∘myfun) => [:a, :b])

```

There’s also an [issue](https://github.com/JuliaData/DataFrames.jl/issues/2549) discussing the idea of making this easier.

---

<div class="post-metadata">

**Author:** ![joonp](https://avatars.discourse-cdn.com/v4/letter/j/a9a28c/32.png) [@joonp](https://discourse.julialang.org/u/joonp)\
**Post date:** [July 3, 2021, 1:09pm UTC](https://discourse.julialang.org/t/how-to-properly-add-two-new-columns-in-dataframe-from-a-function-that-returns-two-arrays/63964/7 "2021-07-03T13:09:47Z")

</div>

Thank you for listing all these excellent alternatives.

One quick follow-up question—  
Is their performance comparable? I wonder if some of them are dominantly better than the other in terms of speed.

---

<div class="post-metadata">

**Author:** ![sijo](https://avatars.discourse-cdn.com/v4/letter/s/da6949/32.png) [@sijo](https://discourse.julialang.org/u/sijo)\
**Post date:** [July 3, 2021, 1:34pm UTC](https://discourse.julialang.org/t/how-to-properly-add-two-new-columns-in-dataframe-from-a-function-that-returns-two-arrays/63964/8 "2021-07-03T13:34:31Z")

</div>

I don’t know… probably best to run benchmarks for your real use case.

---

<div class="post-metadata">

**Author:** ![joonp](https://avatars.discourse-cdn.com/v4/letter/j/a9a28c/32.png) [@joonp](https://discourse.julialang.org/u/joonp)\
**Post date:** [July 3, 2021, 1:42pm UTC](https://discourse.julialang.org/t/how-to-properly-add-two-new-columns-in-dataframe-from-a-function-that-returns-two-arrays/63964/9 "2021-07-03T13:42:09Z")

</div>

Thanks! I will do some tests.