# Where is the input column name information in a dataframe transformation?

**URL:** <https://discourse.julialang.org/t/where-is-the-input-column-name-information-in-a-dataframe-transformation/91973>\
**Category:** Data\
**Tags:** dataframes\
**Created:** [December 21, 2022, 7:55pm UTC](https://discourse.julialang.org/t/where-is-the-input-column-name-information-in-a-dataframe-transformation/91973 "2022-12-21T19:55:11Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![rocco\_sprmnt21](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rocco_sprmnt21/32/20127_2.png) [@rocco\_sprmnt21](https://discourse.julialang.org/u/rocco_sprmnt21)\
**Post date:** [December 21, 2022, 7:55pm UTC](https://discourse.julialang.org/t/where-is-the-input-column-name-information-in-a-dataframe-transformation/91973/1 "2022-12-21T19:55:11Z")

</div>

suppose I want to use some specific columns of a dataframe to do some operations and output another number of columns.  
In dynamically defining the name of these columns I want to use the name of the source columns.

In the following case I can do it because after fun=\> I have the input names available.

```julia
transform(df, Cols([2,3,1,5])=>fun=>x->"new_".*x[4:-1:2])

```

But in the following case if one want to use the information inside the funcols() function to produce a namedtuple for AsTable, how do you do it?

```julia
transform(df, Cols([2,3,1,5])=>funcols=>AsTable)

```

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [December 21, 2022, 9:13pm UTC](https://discourse.julialang.org/t/where-is-the-input-column-name-information-in-a-dataframe-transformation/91973/2 "2022-12-21T21:13:05Z")

</div>

You have to write `funcols` to return a `NamedTuple` and work with the names inside `funcols`. It all happens inside `funcols` and not in the destination.

---

<div class="post-metadata">

**Author:** ![rocco\_sprmnt21](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rocco_sprmnt21/32/20127_2.png) [@rocco\_sprmnt21](https://discourse.julialang.org/u/rocco_sprmnt21)\
**Post date:** [December 21, 2022, 9:30pm UTC](https://discourse.julialang.org/t/where-is-the-input-column-name-information-in-a-dataframe-transformation/91973/3 "2022-12-21T21:30:56Z")

</div>

inside funcols I have the column data but not the names. I wish I had the names available to use in creating the new names.  
I know you can achieve the same result in other ways, but I wanted to know if this is somehow possible as well

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [December 21, 2022, 10:55pm UTC](https://discourse.julialang.org/t/where-is-the-input-column-name-information-in-a-dataframe-transformation/91973/4 "2022-12-21T22:55:48Z")

</div>

No I don’t think so.

```julia
AsTable(Cols([2,3,1,5])) => funcols => AsTable

```

---

<div class="post-metadata">

**Author:** ![rocco\_sprmnt21](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rocco_sprmnt21/32/20127_2.png) [@rocco\_sprmnt21](https://discourse.julialang.org/u/rocco_sprmnt21)\
**Post date:** [December 22, 2022, 10:10am UTC](https://discourse.julialang.org/t/where-is-the-input-column-name-information-in-a-dataframe-transformation/91973/5 "2022-12-22T10:10:43Z")

</div>

I was aware of this possibility. What I’m missing is understanding how (and why) it is possible to refer (within the same transform) to the names of the input columns after the second “=\>” and not also after the first “=\>”.  
Just to better understand some internal mechanisms of DataFRames, not because I have a particular need for this “feature”.

---

<div class="post-metadata">

**Author:** ![sijo](https://avatars.discourse-cdn.com/v4/letter/s/da6949/32.png) [@sijo](https://discourse.julialang.org/u/sijo)\
**Post date:** [December 22, 2022, 12:01pm UTC](https://discourse.julialang.org/t/where-is-the-input-column-name-information-in-a-dataframe-transformation/91973/6 "2022-12-22T12:01:13Z")

</div>

The reason is that with `cols => f => g`, DataFrames will call `f` with the column values, and `g` with the column names. It’s just how the API works. If you want to receive the names in the `f` function you can write `AsTable(cols) => f`. In this case DataFrames will pass to `f` the columns as a named tuple (the keys are the column names, the values are the column values). Note that in this case all the columns are passed as a single argument:

```julia
using DataFrames

df = DataFrame(a=1:3, b=4:6)

function f(table)
    # Add the (first) two columns of table
    result = table[1] + table[2]
    
    # Make the name "a+b" from column names "a" and "b"
    names = keys(table)
    result_name = Symbol(names[1], "+", names[2])
    return (; result_name => result)
end

transform(df, AsTable([:a, :b]) => f => AsTable)

# Output:

3×3 DataFrame
 Row │ a b a+b   
     │ Int64 Int64 Int64 
─────┼─────────────────────
   1 │ 1 4 5
   2 │ 2 5 7
   3 │ 3 6 9

```

---

<div class="post-metadata">

**Author:** ![rocco\_sprmnt21](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rocco_sprmnt21/32/20127_2.png) [@rocco\_sprmnt21](https://discourse.julialang.org/u/rocco_sprmnt21)\
**Post date:** [December 22, 2022, 6:53pm UTC](https://discourse.julialang.org/t/where-is-the-input-column-name-information-in-a-dataframe-transformation/91973/7 "2022-12-22T18:53:47Z")

</div>

As mentioned in previous posts, I know of other ways to get the result, including passing namedtuples via AsTable.

In this case I would do so

```julia
transform(df, [:a, :b] => (+) => x->join(x,"+"))

```

what I was trying to know is if it was somehow possible to use, even when cols=Array{Symbol}, the column names inside the func and not just in the output naming context.

Since after the second “=\>” I can define a function x-\>secondfun(colsnames) that uses the column names, I was trying to imagine IF and HOW I could use the same information (the names) in the context of the first function.  
I know that currently the APIs work the way they do, but I just wanted to poke around behind the scene, without going through the code which is very large and complex.

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [December 22, 2022, 6:58pm UTC](https://discourse.julialang.org/t/where-is-the-input-column-name-information-in-a-dataframe-transformation/91973/8 "2022-12-22T18:58:55Z")

</div>

It seems like we are going in circles. The answer is simply no, the API is not constructed in that way. `AsTable` only knows about the named-tuple passed to it. It doesn’t know about `src` in any way in the `src => fun => dest` expression. There is a way to do what you want, and we have described it above.

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [December 22, 2022, 7:27pm UTC](https://discourse.julialang.org/t/where-is-the-input-column-name-information-in-a-dataframe-transformation/91973/9 "2022-12-22T19:27:57Z")

</div>

You could do a wrapper function around `fun`.

```julia
fix_names(nt, src) = ... fix column names based on src... returns a new named tuple
src => (t -> fix_names(fun(t), src)) => AsTable

```

---

<div class="post-metadata">

**Author:** ![rocco\_sprmnt21](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rocco_sprmnt21/32/20127_2.png) [@rocco\_sprmnt21](https://discourse.julialang.org/u/rocco_sprmnt21)\
**Post date:** [December 22, 2022, 9:01pm UTC](https://discourse.julialang.org/t/where-is-the-input-column-name-information-in-a-dataframe-transformation/91973/10 "2022-12-22T21:01:45Z")

</div>

It looks like Columbus’s egg, but the answer to my question could be like this, except that you have to use an external variable

PS  
We weren’t going in circles but following a spiral we might eventually get to (or get close to) some point

```julia
src=[:a, :b]
aplusb(src, x)= (;zip(Symbol.([join(src,"+")]), [+(x...)])...)
transform(df, src=> ((x...)->aplusb(src,x)) => AsTable)

```