# Transform multiple columns of a DataFrame using the same function

**URL:** <https://discourse.julialang.org/t/transform-multiple-columns-of-a-dataframe-using-the-same-function/61550>\
**Category:** Data\
**Tags:** dataframes\
**Created:** [May 21, 2021, 12:18am UTC](https://discourse.julialang.org/t/transform-multiple-columns-of-a-dataframe-using-the-same-function/61550 "2021-05-21T00:18:05Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![anon71627001](https://avatars.discourse-cdn.com/v4/letter/a/a698b9/32.png) [@anon71627001](https://discourse.julialang.org/u/anon71627001)\
**Post date:** [May 21, 2021, 12:18am UTC](https://discourse.julialang.org/t/transform-multiple-columns-of-a-dataframe-using-the-same-function/61550/1 "2021-05-21T00:18:05Z")

</div>

I’d like to transform several columns “in-place” using the same function. For example, if I have

```julia
using DataFrames
df = DataFrame(a = 1:3, b = 4:6, c = 7:9)

```

and want to double columns `a` and `c` and replace those columns with the doubled ones, I’d like to be able to do something like

`transform!(df, ["a", "c"] => ByRow(x -> 2x) => ["a", "c"])`

so that I end up with

`DataFrame(a = [2, 4, 6], b = 4:6, c = [14, 16, 18])`

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [May 21, 2021, 5:13am UTC](https://discourse.julialang.org/t/transform-multiple-columns-of-a-dataframe-using-the-same-function/61550/2 "2021-05-21T05:13:05Z")

</div>

Just broadcast the pair operator:

```julia
julia> transform!(df, [:a, :c] .=> (x -> 2x) .=> [:a, :c])
3×3 DataFrame
 Row │ a b c     
     │ Int64 Int64 Int64 
─────┼─────────────────────
   1 │ 2 4 14
   2 │ 4 5 16
   3 │ 6 6 18

```

(no need for `ByRow` as you can just double the whole vector).

---

<div class="post-metadata">

**Author:** ![bkamins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bkamins/32/208538_2.png) [@bkamins](https://discourse.julialang.org/u/bkamins)\
**Post date:** [May 21, 2021, 6:00am UTC](https://discourse.julialang.org/t/transform-multiple-columns-of-a-dataframe-using-the-same-function/61550/3 "2021-05-21T06:00:55Z")

</div>

Also if the list of columns is long you can skip specifying the output names and use `renamecols=false` kwarg.

---

<div class="post-metadata">

**Author:** ![rocco\_sprmnt21](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rocco_sprmnt21/32/20127_2.png) [@rocco\_sprmnt21](https://discourse.julialang.org/u/rocco_sprmnt21)\
**Post date:** [May 21, 2021, 10:49am UTC](https://discourse.julialang.org/t/transform-multiple-columns-of-a-dataframe-using-the-same-function/61550/4 "2021-05-21T10:49:33Z")

</div>

If you really want to use ByRow, ,try one of these (I got there by trial and error.  
I obviously haven’t read all the documentation on the transform function yet) ) :

```julia
transform(df, AsTable([:a, :c]) => ByRow(x ->(2x.a,2x.c)) =>[:a,:c])

transform(df, AsTable([:a, :c]) => ByRow(x ->2 .* values(x)) =>[:a,:c])

```

anhoter way

```julia
 transform!(df, [:a, :c] => ByRow((x...) -> 2 .* x)=>[:a,:c])

```

---

<div class="post-metadata">

**Author:** ![nalimilan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nalimilan/32/147_2.png) [@nalimilan](https://discourse.julialang.org/u/nalimilan)\
**Post date:** [May 21, 2021, 11:52am UTC](https://discourse.julialang.org/t/transform-multiple-columns-of-a-dataframe-using-the-same-function/61550/5 "2021-05-21T11:52:57Z")

</div>

This is not a the best solution: these examples pass all the columns to a single function which returns multiple columns. This is more complex than it needs to be and it will trigger more compilation than needed. The best way is to use `.=>`, and it works fine in combination with `ByRow`.

---

<div class="post-metadata">

**Author:** ![anon71627001](https://avatars.discourse-cdn.com/v4/letter/a/a698b9/32.png) [@anon71627001](https://discourse.julialang.org/u/anon71627001)\
**Post date:** [May 21, 2021, 2:49pm UTC](https://discourse.julialang.org/t/transform-multiple-columns-of-a-dataframe-using-the-same-function/61550/6 "2021-05-21T14:49:03Z")

</div>

Thank you for the advice but I wasn’t able to get renamecols to work. Maybe my syntax is wrong:

```julia
df = DataFrame(a = 1:3, b = 4:6, c = 7:9)
transform!(df, [:a, :c] .=> (x -> 2x); renamecols=false)

MethodError: no method matching transform!(::DataFrames.DataFrame, ::Vector{Pair{String, var"#25#26"{typeof(*)}}}; renamecols=false)

Closest candidates are:

transform!(::DataFrames.DataFrame, ::Any...) at /Users/x/.julia/packages/DataFrames/GtZ1l/src/abstractdataframe/selection.jl:371 got unsupported keyword argument "renamecols"

transform!(!Matched::DataFrames.GroupedDataFrame{DataFrames.DataFrame}, ::Any...; ungroup) at /Users/x/.julia/packages/DataFrames/GtZ1l/src/groupeddataframe/splitapplycombine.jl:1719 got unsupported keyword argument "renamecols"

```

---

<div class="post-metadata">

**Author:** ![anon71627001](https://avatars.discourse-cdn.com/v4/letter/a/a698b9/32.png) [@anon71627001](https://discourse.julialang.org/u/anon71627001)\
**Post date:** [May 21, 2021, 2:52pm UTC](https://discourse.julialang.org/t/transform-multiple-columns-of-a-dataframe-using-the-same-function/61550/7 "2021-05-21T14:52:07Z")

</div>

I think ByRow can still work with @nilshg’s reply. This code runs:

```julia
df = DataFrame(a = 1:3, b = 4:6, c = 7:9)
transform!(df, [:a, :c] .=> ByRow(x -> 2x) .=> [:a, :c])

```

---

<div class="post-metadata">

**Author:** ![rocco\_sprmnt21](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rocco_sprmnt21/32/20127_2.png) [@rocco\_sprmnt21](https://discourse.julialang.org/u/rocco_sprmnt21)\
**Post date:** [May 21, 2021, 3:49pm UTC](https://discourse.julialang.org/t/transform-multiple-columns-of-a-dataframe-using-the-same-function/61550/8 "2021-05-21T15:49:36Z")

</div>

Yes, I saw. Thank you.  
I am trying to understand the various ways of acting on the elements of the dataframe.  
For example, I saw [here](https://dataframes.juliadata.org/stable/man/split_apply_combine/) that it is possible to use a vector of triples cols =\> function =\> target\_cols

```julia
colsname=names(df)
f=[x->2x,x->4x]
oddcol=colsname[1:2:3]
transform(df,[oddcol[i]=>ByRow(f[i])=>oddcol[i] for i in 1:length(oddcol)])

```

---

<div class="post-metadata">

**Author:** ![sijo](https://avatars.discourse-cdn.com/v4/letter/s/da6949/32.png) [@sijo](https://discourse.julialang.org/u/sijo)\
**Post date:** [May 21, 2021, 4:00pm UTC](https://discourse.julialang.org/t/transform-multiple-columns-of-a-dataframe-using-the-same-function/61550/9 "2021-05-21T16:00:14Z")

</div>

> [@anon71627001](#):
>
> ```julia
> df = DataFrame(a = 1:3, b = 4:6, c = 7:9)
> transform!(df, [:a, :c] .=> (x -> 2x); renamecols=false)
> 
> ```

This works for me on Julia 1.6 with DataFrames 1.1.1.

---

<div class="post-metadata">

**Author:** ![anon71627001](https://avatars.discourse-cdn.com/v4/letter/a/a698b9/32.png) [@anon71627001](https://discourse.julialang.org/u/anon71627001)\
**Post date:** [May 21, 2021, 5:16pm UTC](https://discourse.julialang.org/t/transform-multiple-columns-of-a-dataframe-using-the-same-function/61550/10 "2021-05-21T17:16:16Z")

</div>

Thank you, I hadn’t updated DataFrames.jl. It works for me now.

---

<div class="post-metadata">

**Author:** ![rocco\_sprmnt21](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rocco_sprmnt21/32/20127_2.png) [@rocco\_sprmnt21](https://discourse.julialang.org/u/rocco_sprmnt21)\
**Post date:** [June 6, 2021, 1:06pm UTC](https://discourse.julialang.org/t/transform-multiple-columns-of-a-dataframe-using-the-same-function/61550/11 "2021-06-06T13:06:35Z")

</div>

I made several tests to achieve the required transformation, including the 3 reported here.  
I would have expected (due to a sort of symmetry between input and output) the third to give the same result, but it doesn’t work.

```julia
transform(df, r"x"=> ((x,y)->(;ax=2x,bx=2y))=>AsTable )
transform(df, r"x".=> ByRow((x...) -> 2 .*x)=>[:ax,:bx])
transform(df, r"x".=> ByRow((x...) -> 2 .*x)=>r"x")

```

while an expression like this works:

```julia
transform(df, r"x"=> ByRow((x...) -> 2 .*x)=>names(df,r"x"))

```

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [June 6, 2021, 1:47pm UTC](https://discourse.julialang.org/t/transform-multiple-columns-of-a-dataframe-using-the-same-function/61550/12 "2021-06-06T13:47:37Z")

</div>

I mean, `src => fun => r"x"` doesn’t have an intuitive meaning. `r"x"` is a selector that only makes sense in the context of a given data frame, which is why we need `names(df, r"x")`.

I think the asymmetry is logical, as `dest` is only about the output of `fun` rather than the data frame as a whole. Think of the first example, with `AsTable`. The `AsTable` knows nothing about the input data frame, only the output of `fun`.

You can also add the keyword argument `renamecols = false` to get the same names as the input.

---

<div class="post-metadata">

**Author:** ![Eben60](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/eben60/32/13475_2.png) [@Eben60](https://discourse.julialang.org/u/Eben60)\
**Post date:** [January 23, 2023, 4:45pm UTC](https://discourse.julialang.org/t/transform-multiple-columns-of-a-dataframe-using-the-same-function/61550/13 "2023-01-23T16:45:38Z")

</div>

For use with regex, the following form is probably the most readable

```julia
df = DataFrame(aa = 1:3, b = 4:6, ac = 7:9)
transform(df, names(df, r"a") .=> x -> 2 .* x ; renamecols=false)

```