# Apply function By Row without re-stating column names

**URL:** <https://discourse.julialang.org/t/apply-function-by-row-without-re-stating-column-names/65765>\
**Category:** General Usage\
**Tags:** dataframes, functions\
**Created:** [August 4, 2021, 12:13am UTC](https://discourse.julialang.org/t/apply-function-by-row-without-re-stating-column-names/65765 "2021-08-04T00:13:53Z")\
**Posts on this page:** 17\
**Page:** 2

<div class="post-metadata">

**Author:** ![rocco\_sprmnt21](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rocco_sprmnt21/32/20127_2.png) [@rocco\_sprmnt21](https://discourse.julialang.org/u/rocco_sprmnt21)\
**Post date:** [February 10, 2022, 9:05pm UTC](https://discourse.julialang.org/t/apply-function-by-row-without-re-stating-column-names/65765/21 "2022-02-10T21:05:49Z")

</div>

Just a slight variation of what is already present in this very interesting discussion

```julia

function f2(t) 
    x = t.a + t.b    
    y = t.a - t.b
    (;x,y, t...)
end

select(D, AsTable(:)=>ByRow(f2)=>AsTable)

```

this variant if you need speed

```julia
DataFrame(map(f2, Tables.namedtupleiterator(D)))

```

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [February 10, 2022, 9:25pm UTC](https://discourse.julialang.org/t/apply-function-by-row-without-re-stating-column-names/65765/22 "2022-02-10T21:25:04Z")

</div>

This works on master, but we have not made a release yet with that update. I wanted to add one more feature, support for keyword arguments, but that is taking longer than I thought.

---

<div class="post-metadata">

**Author:** ![Lincoln\_Hannah](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lincoln_hannah/32/19198_2.png) [@Lincoln\_Hannah](https://discourse.julialang.org/u/Lincoln_Hannah)\
**Post date:** [February 10, 2022, 10:28pm UTC](https://discourse.julialang.org/t/apply-function-by-row-without-re-stating-column-names/65765/23 "2022-02-10T22:28:11Z")

</div>

This is what I love about Julia. Always a new feature I can’t wait to see 🙂

---

<div class="post-metadata">

**Author:** ![Lincoln\_Hannah](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lincoln_hannah/32/19198_2.png) [@Lincoln\_Hannah](https://discourse.julialang.org/u/Lincoln_Hannah)\
**Post date:** [February 11, 2022, 1:20am UTC](https://discourse.julialang.org/t/apply-function-by-row-without-re-stating-column-names/65765/24 "2022-02-11T01:20:21Z")

</div>

What about an @rtransform equivalent of `=> AsTable`  
So the columns added or altered are given by the items of the named tuple returned by the function.

```julia
function myfunc(; a, b, kwargs... )
    x = a+b
    y = a-b
    (;x, y)
end

@chain begin
    DataFrame( a=1:2, b=3:4 )

    @rtransform :AsTable = myfunc(; AsTable(:)... )
end

```

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [February 11, 2022, 3:04am UTC](https://discourse.julialang.org/t/apply-function-by-row-without-re-stating-column-names/65765/25 "2022-02-11T03:04:35Z")

</div>

That exists! Just do

```julia
@rtransform $AsTable = mufunc(...)

```

It’s insufficiently documented, which is changing in the future. But there is an example in the docs [here](https://juliadata.github.io/DataFramesMeta.jl/stable/#@combine).

---

<div class="post-metadata">

**Author:** ![Lincoln\_Hannah](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lincoln_hannah/32/19198_2.png) [@Lincoln\_Hannah](https://discourse.julialang.org/u/Lincoln_Hannah)\
**Post date:** [February 11, 2022, 3:53am UTC](https://discourse.julialang.org/t/apply-function-by-row-without-re-stating-column-names/65765/26 "2022-02-11T03:53:08Z")

</div>

very cool, thank you 🙂

---

<div class="post-metadata">

**Author:** ![Lincoln\_Hannah](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lincoln_hannah/32/19198_2.png) [@Lincoln\_Hannah](https://discourse.julialang.org/u/Lincoln_Hannah)\
**Post date:** [February 11, 2022, 6:58am UTC](https://discourse.julialang.org/t/apply-function-by-row-without-re-stating-column-names/65765/27 "2022-02-11T06:58:38Z")

</div>

I notice you can’t use `$AsTable =` within an `@rtransform @astable` block.

This would be a nice feature.

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [February 11, 2022, 2:41pm UTC](https://discourse.julialang.org/t/apply-function-by-row-without-re-stating-column-names/65765/28 "2022-02-11T14:41:37Z")

</div>

Can you clarify what that would look like? Do you mean merging selected names and the programmatically generated names from `$AsTable`? Fwiw I think this would be a very hard feature to implement, so I’m interested to hear your use case and proposal.

---

<div class="post-metadata">

**Author:** ![Lincoln\_Hannah](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lincoln_hannah/32/19198_2.png) [@Lincoln\_Hannah](https://discourse.julialang.org/u/Lincoln_Hannah)\
**Post date:** [February 14, 2022, 6:27am UTC](https://discourse.julialang.org/t/apply-function-by-row-without-re-stating-column-names/65765/29 "2022-02-14T06:27:33Z")

</div>

In the below example, column B is added directly, then `myfunc` with `$AsTable` adds columns C and D then column E is added directly.

In my actual code, 5 or 10 columns are added each time. Because `$AsTable` doesn’t work within an `@ratransform @astable` block I have to do it as 3 seperate `@rtransform` blocks.

I’d prefer all the row level operations on the table to be within a single `@rtransform` block.

```julia
function myfunc(; A, B )
    C = A + B
    D = A - B
    (; C, D)
end

@chain begin
    DataFrame( A=1:2 )

    @rtransform @astable begin
        :B = :A + 1      
        $AsTable = myfunc(; AsTable... )
        :E = :C + :D
    end
end

```

---

<div class="post-metadata">

**Author:** ![Lincoln\_Hannah](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lincoln_hannah/32/19198_2.png) [@Lincoln\_Hannah](https://discourse.julialang.org/u/Lincoln_Hannah)\
**Post date:** [February 17, 2022, 12:36am UTC](https://discourse.julialang.org/t/apply-function-by-row-without-re-stating-column-names/65765/30 "2022-02-17T00:36:25Z")

</div>

A more extreme idea (probably too difficult). A block similar to @chain but allows for row level operations (as if you are within @rtransform @astable) except where other macros ( e.g. @rsubset, @orderby) are being used.

```julia
@ChainWithrTransform DataFrame( A = 1:10 ) begin

    :B = mod(:A,3)
    :E = :B * 2

    @rsubset :B == 1
    
    :F = :B * 2

    @orderby :A

    $AsTable = myfunc(; AsTable... )

    :G = :D * 2

end

```

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [February 17, 2022, 2:04pm UTC](https://discourse.julialang.org/t/apply-function-by-row-without-re-stating-column-names/65765/31 "2022-02-17T14:04:16Z")

</div>

Maybe. then this new `@chain`-like macro would have to live in DataFramesMeta.jl, rather than Chain.jl.

Another consideration is what `:B = mod(:A, 3)` would do. It seems costly to copy a new data frame every time, maybe it could do `@rtransform!` behind the scenes?

Can you please file an issue?

---

<div class="post-metadata">

**Author:** ![Lincoln\_Hannah](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lincoln_hannah/32/19198_2.png) [@Lincoln\_Hannah](https://discourse.julialang.org/u/Lincoln_Hannah)\
**Post date:** [February 17, 2022, 11:27pm UTC](https://discourse.julialang.org/t/apply-function-by-row-without-re-stating-column-names/65765/32 "2022-02-17T23:27:20Z")

</div>

[https://github.com/JuliaData/DataFramesMeta.jl/issues/325](https://github.com/JuliaData/DataFramesMeta.jl/issues/325)

---

<div class="post-metadata">

**Author:** ![Lincoln\_Hannah](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lincoln_hannah/32/19198_2.png) [@Lincoln\_Hannah](https://discourse.julialang.org/u/Lincoln_Hannah)\
**Post date:** [April 13, 2022, 5:46am UTC](https://discourse.julialang.org/t/apply-function-by-row-without-re-stating-column-names/65765/33 "2022-04-13T05:46:43Z")

</div>

has this been released now?

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [April 15, 2022, 7:48pm UTC](https://discourse.julialang.org/t/apply-function-by-row-without-re-stating-column-names/65765/34 "2022-04-15T19:48:35Z")

</div>

Sorry about that! I was trying to get another feature done before a release, but I should just push it out now. Let me add a `NEWS.md` entry and then I will make a release.

---

<div class="post-metadata">

**Author:** ![Lincoln\_Hannah](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lincoln_hannah/32/19198_2.png) [@Lincoln\_Hannah](https://discourse.julialang.org/u/Lincoln_Hannah)\
**Post date:** [May 8, 2022, 11:34pm UTC](https://discourse.julialang.org/t/apply-function-by-row-without-re-stating-column-names/65765/35 "2022-05-08T23:34:18Z")

</div>

Just a thought.

A function within a @chain block applies to the previous line’s output. e.g. the below gives 15.

```julia
@chain 1:5 begin
     sum
end  

```

If the previous line’s result is a DataFrame and the function’s input and output are both named tuples, could the function be automatically applied at row level to add columns to the DataFrame ?  
with example data.

```julia
function my_func(; a, b ) 
        c = a + b
        d = a - b
       (; c, d ) 
end

DF = DataFrame( a = 1:3, b=4:6 )

```

Could this

```julia
@chain DF begin
       my_func
end

```

be equivalent to this

```julia
@chain DF begin
       @rtransform $AsTable = my_func(; AsTable... )
end

```

to produce a DataFrame with columns a, b, c, d.

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [May 9, 2022, 6:05am UTC](https://discourse.julialang.org/t/apply-function-by-row-without-re-stating-column-names/65765/36 "2022-05-09T06:05:47Z")

</div>

No, this is impossible. In the expression

```julia
@chain DF begin
     my_func
end

```

`@chain`, the macro, does not know anything about `my_func` and thus can’t produce that complicated expression based only on `my_func`.

But I hear your concern that your method needs a lot of typing. I will think about ways to make that easier.

---

<div class="post-metadata">

**Author:** ![Lincoln\_Hannah](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lincoln_hannah/32/19198_2.png) [@Lincoln\_Hannah](https://discourse.julialang.org/u/Lincoln_Hannah)\
**Post date:** [May 9, 2022, 7:01am UTC](https://discourse.julialang.org/t/apply-function-by-row-without-re-stating-column-names/65765/37 "2022-05-09T07:01:22Z")

</div>

Thanks Peter. much appreciated.

unrelated question -

In Pandas you can select rows using a specified index / key column.  
Does Julia have an equivalent ? I haven’t been able to find it.

It would be very nice within a @transform block to specify a dataframe cell as :RowKey.Colum  
In the below, :C3 would be 12,13, 14. i.e. the values of :C2 plus the :C1 value when RowKey == :A.

```julia
df = DataFrame( KeyCol = [:A,:B,:C], C1 = 1:3, C2 = 11:13 )

setKeyColumn!(df, :KeyCol )

@chain df begin  

    @transform :C3 = :C2 .+ :A.C1

end

```

[Previous page](https://discourse.julialang.org/t/apply-function-by-row-without-re-stating-column-names/65765.md?page=1)
