# Efficient way to add column to dataframe computed from prior columns

**URL:** <https://discourse.julialang.org/t/efficient-way-to-add-column-to-dataframe-computed-from-prior-columns/66290>\
**Category:** New to Julia\
**Tags:** question\
**Created:** [August 12, 2021, 7:40pm UTC](https://discourse.julialang.org/t/efficient-way-to-add-column-to-dataframe-computed-from-prior-columns/66290 "2021-08-12T19:40:12Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![claylyons](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/claylyons/32/27450_2.png) [@claylyons](https://discourse.julialang.org/u/claylyons)\
**Post date:** [August 12, 2021, 7:40pm UTC](https://discourse.julialang.org/t/efficient-way-to-add-column-to-dataframe-computed-from-prior-columns/66290/1 "2021-08-12T19:40:12Z")

</div>

I was struggling to find some clean way to code adding a named column to an existing dataframe computed from some of its columns. While I started posting this question I stumbled upon the official docs description of combine/transform and I was quite happy with using transform as posted below. But still, I am wondering if there are other ways to code this operation that are clean and possibly more performant(cpu usage/memory).

Also, does anybody have a good list of nice/terse/clean julia formulas for creating and transforming dataframes? What I am looking for is really a bunch of code that I can try out in the julia repl to get a better understanding of the syntax.

```julia
julia> df = DataFrame(X = [1, 2, 3, 4], Y = [0, 1, 2, 4])
4×2 DataFrame
 Row │ X Y     
     │ Int64 Int64 
─────┼──────────────
   1 │ 1 0
   2 │ 2 1
   3 │ 3 2
   4 │ 4 4

julia> add = (a, b) -> a + b
#7 (generic function with 1 method)

julia> transform!(df, :, [:X, :Y] => add => :Z)
4×3 DataFrame
 Row │ X Y Z     
     │ Int64 Int64 Int64 
─────┼─────────────────────
   1 │ 1 0 1
   2 │ 2 1 3
   3 │ 3 2 5
   4 │ 4 4 8

```

---

<div class="post-metadata">

**Author:** ![stillyslalom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stillyslalom/32/45687_2.png) [@stillyslalom](https://discourse.julialang.org/u/stillyslalom)\
**Post date:** [August 12, 2021, 8:19pm UTC](https://discourse.julialang.org/t/efficient-way-to-add-column-to-dataframe-computed-from-prior-columns/66290/2 "2021-08-12T20:19:02Z")

</div>

You can do `df.Z = df.X + df.Y`. For more complicated operations, I like Chain.jl plus either DataFrameMacros.jl or DataFramesMeta.jl (both provide similar tools, but the former operates by-row while the latter operates on columns by default). For example,

```julia
julia> using DataFrameMacros, Chain

julia> @chain df begin
           @transform!(:Z = add(:X, :Y))
       end
4×3 DataFrame
 Row │ X Y Z
     │ Int64 Int64 Int64
─────┼─────────────────────
   1 │ 1 0 1
   2 │ 2 1 3
   3 │ 3 2 5
   4 │ 4 4 8

```

If you don’t need to chain multiple operations together, you can omit `@chain`:

```julia
julia> @transform!(df, :Z = add(:X, :Y))
4×3 DataFrame
 Row │ X Y Z
     │ Int64 Int64 Int64
─────┼─────────────────────
   1 │ 1 0 1
   2 │ 2 1 3
   3 │ 3 2 5
   4 │ 4 4 8

```

---

<div class="post-metadata">

**Author:** ![claylyons](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/claylyons/32/27450_2.png) [@claylyons](https://discourse.julialang.org/u/claylyons)\
**Post date:** [August 12, 2021, 8:31pm UTC](https://discourse.julialang.org/t/efficient-way-to-add-column-to-dataframe-computed-from-prior-columns/66290/3 "2021-08-12T20:31:19Z")

</div>

I was hoping there was a way to express this in a more declarative fashion like you have done!

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [August 12, 2021, 8:33pm UTC](https://discourse.julialang.org/t/efficient-way-to-add-column-to-dataframe-computed-from-prior-columns/66290/4 "2021-08-12T20:33:16Z")

</div>

> [@stillyslalom](#):
>
> (both provide similar tools, but the former operates by-row while the latter operates on columns by default)

Note, DataFramesMeta.jl now exports the macros `@rtransform`, `@rselect`, `@rsubset`, and `@rorderby`. So it now has feature parity for row-wise operations (with the addition of the letter `r`).

```julia
using DataFramesMeta # also exports Chain.jl
@chain df begin 
    @rtransform! :Z = :X + :Y
end

```

---

<div class="post-metadata">

**Author:** ![jzr](https://avatars.discourse-cdn.com/v4/letter/j/eb9ed0/32.png) [@jzr](https://discourse.julialang.org/u/jzr)\
**Post date:** [August 12, 2021, 8:33pm UTC](https://discourse.julialang.org/t/efficient-way-to-add-column-to-dataframe-computed-from-prior-columns/66290/5 "2021-08-12T20:33:18Z")

</div>

Personally I prefer the non-mutating form `transform` over the mutating `transform!` and the base DataFrames library rather than macros. Both of these choices simplify my code.

And I’m another +1 for Chain.jl.

---

<div class="post-metadata">

**Author:** ![stillyslalom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stillyslalom/32/45687_2.png) [@stillyslalom](https://discourse.julialang.org/u/stillyslalom)\
**Post date:** [August 12, 2021, 8:58pm UTC](https://discourse.julialang.org/t/efficient-way-to-add-column-to-dataframe-computed-from-prior-columns/66290/6 "2021-08-12T20:58:12Z")

</div>

It might be nice to mention the ‘declarative’ form more prominently in the docs. As far as I can tell, it’s mostly described [here](https://dataframes.juliadata.org/latest/man/getting_started/#Constructing-Column-by-Column) as a way to create a `DataFrame` from scratch. The docstring for `transform!` is ~1600 words long and applies to four distinct methods, includes many conditional clauses, and has no examples, which may be a bit intimidating for a newcomer. It may be worth including an example in the `transform!` docs that uses both `df.Z = df.X + df.Y` and `transform!(df, :, [:X, :Y] => + => :Z)` to intercept users before they get buried in contemplation of which of the seven allowable forms of `args...` to apply.

---

<div class="post-metadata">

**Author:** ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Post date:** [August 12, 2021, 11:59pm UTC](https://discourse.julialang.org/t/efficient-way-to-add-column-to-dataframe-computed-from-prior-columns/66290/7 "2021-08-12T23:59:54Z")

</div>

> [@claylyons](#):
>
> Also, does anybody have a good list of nice/terse/clean julia formulas for creating and transforming

This video might be relevant: [https://www.youtube.com/watch?v=rDvpLFxcL84](https://www.youtube.com/watch?v=rDvpLFxcL84)

But I think DataFrameMacros.jl and DataFramesMeta.jl are both good for what you want.

I think it’s a good decision for DataFrames.jl to remain macro-less cos then macro convenience packages can build various API that people can use.