# How to use the transform function on data frames

**URL:** <https://discourse.julialang.org/t/how-to-use-the-transform-function-on-data-frames/104582>\
**Category:** General Usage\
**Tags:** dataframes\
**Created:** [October 4, 2023, 7:33pm UTC](https://discourse.julialang.org/t/how-to-use-the-transform-function-on-data-frames/104582 "2023-10-04T19:33:05Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![RobertTalbert](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/roberttalbert/32/202249_2.png) [@RobertTalbert](https://discourse.julialang.org/u/RobertTalbert)\
**Post date:** [October 4, 2023, 7:33pm UTC](https://discourse.julialang.org/t/how-to-use-the-transform-function-on-data-frames/104582/1 "2023-10-04T19:33:05Z")

</div>

I apologize for a very low-level question but I’ve just started learning the language and getting stumped. I am trying to take a dataframe and create a new column that’s the average of two other columns. I understand that this is what the `transform` function is supposed to do, but I’m having a hard time understanding the syntax. I entered the following:

```julia
using DataFrames
df = DataFrame(A = [10, 20, 30, 40, 50], B = [5, 15, 25, 35, 45])
df = transform(df, :Average => ((df.A + df.B) / 2) => :Average)
println(df)

```

What I expected was for there for be a new column “Average” that has the average of columns A and B. But I got the error message: **ArgumentError: invalid index: :Average =\> ([7.5, 17.5, 27.5, 37.5, 47.5] =\> :Average) of type Pair{Symbol, Pair{Vector{Float64}, Symbol}}** I’m not able to make anything of that message.

Thanks for your patience and help!

---

<div class="post-metadata">

**Author:** ![ginger\_snaps](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ginger_snaps/32/203688_2.png) [@ginger\_snaps](https://discourse.julialang.org/u/ginger_snaps)\
**Post date:** [October 4, 2023, 8:06pm UTC](https://discourse.julialang.org/t/how-to-use-the-transform-function-on-data-frames/104582/2 "2023-10-04T20:06:23Z")

</div>

Why not just  
`df.Average = (df.A + df.B) / 2`  
?

---

<div class="post-metadata">

**Author:** ![bertschi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bertschi/32/33462_2.png) [@bertschi](https://discourse.julialang.org/u/bertschi)\
**Post date:** [October 4, 2023, 8:10pm UTC](https://discourse.julialang.org/t/how-to-use-the-transform-function-on-data-frames/104582/3 "2023-10-04T20:10:35Z")

</div>

A way to use `transform` is the following:

```julia
transform(df, [:A, :B] => ((a, b) -> (a .+ b) ./ 2) => :Average)

```

i.e., you need to provide `original_columns => some_function => new_column`. Note that the function takes the whole column vectors as arguments and usually requires explicit broadcasting.

`DataFramesMeta` provides a somewhat nicer syntax, allowing to access old and new columns by name directly:

```julia
@transform df :Avg = (:A .+ :B) ./ 2

```

Note that this expand exactly to the above `transform` call as you may check via `macroexpand`.

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [October 4, 2023, 8:11pm UTC](https://discourse.julialang.org/t/how-to-use-the-transform-function-on-data-frames/104582/4 "2023-10-04T20:11:53Z")

</div>

In the `source => fun => dest` syntax you have

1. The `source`: This is a column or a collection of columns. You got this wrong, since you want the input to be two columns, `:A`, and `:B`
2. The `fun`: This is a _function_. Frequently, people put anonymous functions here, as in `(a, b) -> a + b)`
3. The `dest`: This is the name of the new column being created. You got this right with `:Average`.

Solution 1:

```julia
# ByRow because this is row-wise
df = transform(df, [:A, :B] => ByRow((A, B) -> (A + B) / 2)) => :Average)

```

Solution 2: Use the package DataFramesMeta.jl, which wraps `transform` with a nicer syntax

```julia
df = @transform df :Average = (:A + :B) / 2

```

---

<div class="post-metadata">

**Author:** ![RobertTalbert](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/roberttalbert/32/202249_2.png) [@RobertTalbert](https://discourse.julialang.org/u/RobertTalbert)\
**Post date:** [October 4, 2023, 8:14pm UTC](https://discourse.julialang.org/t/how-to-use-the-transform-function-on-data-frames/104582/5 "2023-10-04T20:14:36Z")

</div>

Thanks to the first three comments here, all of these seem to work fine. I wasn’t aware of the first approach (`df.Average = (df.A + df.B)/2`). I’ll look into the DataFramesMeta package. These are all solutions but I’d like to keep the thread rolling so I can see different ways of doing this, thanks.

---

<div class="post-metadata">

**Author:** ![Nathan\_Boyer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nathan_boyer/32/14825_2.png) [@Nathan\_Boyer](https://discourse.julialang.org/u/Nathan_Boyer)\
**Post date:** [October 4, 2023, 8:43pm UTC](https://discourse.julialang.org/t/how-to-use-the-transform-function-on-data-frames/104582/6 "2023-10-04T20:43:36Z")

</div>

I’ve written a guide on how to use `transform` (and similar) oriented toward Julia beginners which should be included in the next release of the DataFrames.jl documentation. You can preview it [here](https://github.com/JuliaData/DataFrames.jl/blob/6e7ed849fdd90287e84cf5a38366b0a3469a16a0/docs/src/man/manipulation_functions.md). Bogumil also has a tutorial written in notebooks [here](https://github.com/bkamins/Julia-DataFrames-Tutorial) that you may find useful.

(There is currently a Documenter.jl error in my [pull request](https://github.com/JuliaData/DataFrames.jl/pull/3360) preventing merge if anyone wants to help fix.)

---

<div class="post-metadata">

**Author:** ![RobertTalbert](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/roberttalbert/32/202249_2.png) [@RobertTalbert](https://discourse.julialang.org/u/RobertTalbert)\
**Post date:** [October 5, 2023, 4:01pm UTC](https://discourse.julialang.org/t/how-to-use-the-transform-function-on-data-frames/104582/7 "2023-10-05T16:01:21Z")

</div>

Thanks! I’ll check these out.