# Elegant ways to broadcast the same function to each column replacing the original column in DataFrames.jl

**URL:** <https://discourse.julialang.org/t/elegant-ways-to-broadcast-the-same-function-to-each-column-replacing-the-original-column-in-dataframes-jl/61632>\
**Category:** New to Julia\
**Tags:** dataframes\
**Created:** [May 22, 2021, 10:44am UTC](https://discourse.julialang.org/t/elegant-ways-to-broadcast-the-same-function-to-each-column-replacing-the-original-column-in-dataframes-jl/61632 "2021-05-22T10:44:36Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Post date:** [May 22, 2021, 10:44am UTC](https://discourse.julialang.org/t/elegant-ways-to-broadcast-the-same-function-to-each-column-replacing-the-original-column-in-dataframes-jl/61632/1 "2021-05-22T10:44:36Z")

</div>

I can apply a funciton to each column by broadcasting it. BUt I find the syntax of

```julia
@chain df begin
    transform([:a, :b, :c] .=> (x->fn.(x)) .=> [:a, :b, :c])
end

```

clunky. I dont’ like `(x->fn.(x))` in particular as I feel it’s somewhat inelegant. Just looking to see if there are better options.

Full MWE:

```julia
using Chain, DataFrames
df = DataFrame(a = 1:3, b=1:3, c=1:3, d = ["a", "b", "c"]

fn(x) = 2x

@chain df begin
    transform([:a, :b, :c] .=> (x->fn.(x)) .=> [:a, :b, :c])
end

```

---

<div class="post-metadata">

**Author:** ![bkamins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bkamins/32/208538_2.png) [@bkamins](https://discourse.julialang.org/u/bkamins)\
**Post date:** [May 22, 2021, 10:46am UTC](https://discourse.julialang.org/t/elegant-ways-to-broadcast-the-same-function-to-each-column-replacing-the-original-column-in-dataframes-jl/61632/2 "2021-05-22T10:46:23Z")

</div>

This is a point of introducing `ByRow`:

```julia
julia> transform(df, [:a, :b, :c] .=> ByRow(fn) .=> [:a, :b, :c])
3×4 DataFrame
 Row │ a b c d      
     │ Int64 Int64 Int64 String 
─────┼─────────────────────────────
   1 │ 2 2 2 a
   2 │ 4 4 4 b
   3 │ 6 6 6 c

julia> transform(df, [:a, :b, :c] .=> ByRow(fn), renamecols=false)
3×4 DataFrame
 Row │ a b c d      
     │ Int64 Int64 Int64 String 
─────┼─────────────────────────────
   1 │ 2 2 2 a
   2 │ 4 4 4 b
   3 │ 6 6 6 c

```

(as a side benefit having `ByRow` reduces compilation latency)

---

<div class="post-metadata">

**Author:** ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Post date:** [May 22, 2021, 10:48am UTC](https://discourse.julialang.org/t/elegant-ways-to-broadcast-the-same-function-to-each-column-replacing-the-original-column-in-dataframes-jl/61632/3 "2021-05-22T10:48:39Z")

</div>

> [@bkamins](#):
>
> This is a point of introducing `ByRow` :

I was a little confused by `ByRow`. Does it have inefficiencies or should I use it? Was confused by it for a while what is meant by “ByRow”

---

<div class="post-metadata">

**Author:** ![sijo](https://avatars.discourse-cdn.com/v4/letter/s/da6949/32.png) [@sijo](https://discourse.julialang.org/u/sijo)\
**Post date:** [May 22, 2021, 12:12pm UTC](https://discourse.julialang.org/t/elegant-ways-to-broadcast-the-same-function-to-each-column-replacing-the-original-column-in-dataframes-jl/61632/4 "2021-05-22T12:12:30Z")

</div>

To do it for all columns you can simply use `mapcols`.

```julia
julia> df = DataFrame(a = 1:3, b=1:3, c=1:3);

julia> fn(x) = 2x;

julia> mapcols(fn, df)
3×3 DataFrame
 Row │ a b c     
     │ Int64 Int64 Int64 
─────┼─────────────────────
   1 │ 2 2 2
   2 │ 4 4 4
   3 │ 6 6 6

```

(Here I had to remove the `d` column since `fn` cannot be applied to it. I guess you’re aware of `mapcols` and added the `d` column on purpose, but I thought it’d be nice to mention `mapcols` for others reading this thread).

---

<div class="post-metadata">

**Author:** ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Post date:** [May 22, 2021, 12:14pm UTC](https://discourse.julialang.org/t/elegant-ways-to-broadcast-the-same-function-to-each-column-replacing-the-original-column-in-dataframes-jl/61632/5 "2021-05-22T12:14:48Z")

</div>

> [@sijo](#):
>
> for all columns

I just wanted to do to some cols

---

<div class="post-metadata">

**Author:** ![bkamins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bkamins/32/208538_2.png) [@bkamins](https://discourse.julialang.org/u/bkamins)\
**Post date:** [May 22, 2021, 1:14pm UTC](https://discourse.julialang.org/t/elegant-ways-to-broadcast-the-same-function-to-each-column-replacing-the-original-column-in-dataframes-jl/61632/6 "2021-05-22T13:14:29Z")

</div>

This is not the same as what @xiaodai wanted as in this case `fn` is not broadcasted.

---

<div class="post-metadata">

**Author:** ![bkamins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bkamins/32/208538_2.png) [@bkamins](https://discourse.julialang.org/u/bkamins)\
**Post date:** [May 22, 2021, 1:18pm UTC](https://discourse.julialang.org/t/elegant-ways-to-broadcast-the-same-function-to-each-column-replacing-the-original-column-in-dataframes-jl/61632/7 "2021-05-22T13:18:36Z")

</div>

> [@xiaodai](#):
>
> Does it have inefficiencies or should I use it? Was confused by it for a while what is meant by “ByRow”

Simple answer: `ByRow(f)(x...)` is the same as `f.(x...)`.

Complex answer:

- it is the same in most of the cases provided you pass vectors as arguments;
- however, internally we do not use broadcasting, because broadcasting is expensive to compile;
- additionally if `x` is a `NamedTuple` that is a Tables.jl table we use a bit different rule (preserving column names for use inside the function)

The exact rules are super simple:

```julia
(f::ByRow)(cols::AbstractVector...) = map(f.fun, cols...)
(f::ByRow)(table::NamedTuple) = [f.fun(nt) for nt in Tables.namedtupleiterator(table)]

```

---

<div class="post-metadata">

**Author:** ![sijo](https://avatars.discourse-cdn.com/v4/letter/s/da6949/32.png) [@sijo](https://discourse.julialang.org/u/sijo)\
**Post date:** [May 22, 2021, 2:52pm UTC](https://discourse.julialang.org/t/elegant-ways-to-broadcast-the-same-function-to-each-column-replacing-the-original-column-in-dataframes-jl/61632/8 "2021-05-22T14:52:22Z")

</div>

> [@bkamins](#):
>
> This is not the same as what @xiaodai wanted as in this case `fn` is not broadcasted.

Ah good point, it only works in this example because `fn(x) = 2x` and scalar multiplication on columns is equivalent to multiplying row by row.

---

<div class="post-metadata">

**Author:** ![bkamins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bkamins/32/208538_2.png) [@bkamins](https://discourse.julialang.org/u/bkamins)\
**Post date:** [May 22, 2021, 3:10pm UTC](https://discourse.julialang.org/t/elegant-ways-to-broadcast-the-same-function-to-each-column-replacing-the-original-column-in-dataframes-jl/61632/9 "2021-05-22T15:10:08Z")

</div>

Yes, however, still I think `ByRow` is a little easier to read in:

```julia
julia> using DataFrames

julia> df = DataFrame(rand(3,4), :auto)
3×4 DataFrame
 Row │ x1 x2 x3 x4
     │ Float64 Float64 Float64 Float64
─────┼──────────────────────────────────────────
   1 │ 0.798251 0.788097 0.12371 0.447479
   2 │ 0.320884 0.561217 0.736315 0.113512
   3 │ 0.691074 0.807812 0.31742 0.00395885

julia> mapcols(ByRow(sin), df)
3×4 DataFrame
 Row │ x1 x2 x3 x4
     │ Float64 Float64 Float64 Float64
─────┼──────────────────────────────────────────
   1 │ 0.716137 0.709013 0.123395 0.432694
   2 │ 0.315405 0.532217 0.671562 0.113268
   3 │ 0.637365 0.722777 0.312116 0.00395884

julia> mapcols(x -> sin.(x), df)
3×4 DataFrame
 Row │ x1 x2 x3 x4
     │ Float64 Float64 Float64 Float64
─────┼──────────────────────────────────────────
   1 │ 0.716137 0.709013 0.123395 0.432694
   2 │ 0.315405 0.532217 0.671562 0.113268
   3 │ 0.637365 0.722777 0.312116 0.00395884

```

per your proposal

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [May 22, 2021, 4:36pm UTC](https://discourse.julialang.org/t/elegant-ways-to-broadcast-the-same-function-to-each-column-replacing-the-original-column-in-dataframes-jl/61632/10 "2021-05-22T16:36:37Z")

</div>

It will be fast, as fast as `map`. The only exception is with `AsTable` in which case because it acts on a `NamedTupleIterator` there might be some overhead.