# Run multiple instances of transform on specific column combinations of a GroupedDataFrame in DataFrames mini language
**URL:** https://discourse.julialang.org/t/run-multiple-instances-of-transform-on-specific-column-combinations-of-a-groupeddataframe-in-dataframes-mini-language/91736
**Category:** New to Julia
**Tags:** question, dataframes
**Created:** [December 16, 2022, 2:08pm UTC](https://discourse.julialang.org/t/run-multiple-instances-of-transform-on-specific-column-combinations-of-a-groupeddataframe-in-dataframes-mini-language/91736 "2022-12-16T14:08:32Z")
**Posts on this page:** 3
**Page:** 2
### Author:  [@phantom](https://discourse.julialang.org/u/phantom)
#### Post date: [December 23, 2022, 10:14am UTC](https://discourse.julialang.org/t/run-multiple-instances-of-transform-on-specific-column-combinations-of-a-groupeddataframe-in-dataframes-mini-language/91736/21 "2022-12-23T10:14:19Z")
Thank you so much! Sorry I would have liked to mark multiple solutions but wasn’t sure how to do it. If you have a moment could you clarify how the `zip` and `...` commands are working here?
When I add the splat `...` operator after the `Cols` argument in the function declaration does that turn that argument in the function into an array? So when I input `Cols...` into `funnynt` it would be in the form `[col1,col2.col3]`
Also I am not sure what I am getting wrong here. But when I try
```julia
df = DataFrame(id=rand(1:3,10), A=rand(11:13,10),
B=rand(101:103,10), C = rand(25:27,10), D = rand(32:34,10))
10×5 DataFrame
Row │ id A B C D
│ Int64 Int64 Int64 Int64 Int64
─────┼───────────────────────────────────
1 │ 1 12 102 26 32
2 │ 1 13 101 25 34
3 │ 2 13 102 26 33
4 │ 3 11 102 27 33
5 │ 1 13 102 26 33
6 │ 3 11 101 25 32
7 │ 1 11 103 26 34
8 │ 2 12 101 27 34
9 │ 2 12 102 25 32
10 │ 2 12 101 25 32
```
```julia
gdf = groupby(df,:id)
GroupedDataFrame with 3 groups based on key: id
First Group (4 rows): id = 1
Row │ id A B C D
│ Int64 Int64 Int64 Int64 Int64
─────┼───────────────────────────────────
1 │ 1 12 102 26 32
2 │ 1 13 101 25 34
3 │ 1 13 102 26 33
4 │ 1 11 103 26 34
⋮
Last Group (2 rows): id = 3
Row │ id A B C D
│ Int64 Int64 Int64 Int64 Int64
─────┼───────────────────────────────────
1 │ 3 11 102 27 33
2 │ 3 11 101 25 32
```
```julia
function funnynt(id,Cols...; p1=1,p2=2,p3=3)
m= Matrix{Float64}(undef, length(id),length(Cols))
for i in eachindex(Cols)
if p3>0
m[:,i]= (id .+p1).*(Cols[i].+p2*p3)
else
m[:,i]= (id .+p1).*(Cols[i].+p2)
end
end
ncols=Symbol.("id_",names(df,Not(:id)))
return (;zip(ncols,eachcol(m))...)
end
funnynt (generic function with 1 method)
transform(gdf,Cols(:) => funnynt => AsTable)
```
I get
```julia
ERROR: NamedTuple names and field types must have matching lengths
```
Again I apologize if I am overlooking something elementary, but thanks so much for your time and help.
---
### Author:  [@rocco\_sprmnt21](https://discourse.julialang.org/u/rocco_sprmnt21)
#### Post date: [December 23, 2022, 10:57am UTC](https://discourse.julialang.org/t/run-multiple-instances-of-transform-on-specific-column-combinations-of-a-groupeddataframe-in-dataframes-mini-language/91736/22 "2022-12-23T10:57:41Z")
Here my log, starting from a clean session.
See if you can see any difference.
```julia
julia> using DataFrames
julia> df = DataFrame(id=rand(1:3,10), A=rand(11:13,10),
B=rand(101:103,10), C = rand(25:27,10), D = rand(32:34,10))
10×5 DataFrame
Row │ id A B C D
│ Int64 Int64 Int64 Int64 Int64
─────┼───────────────────────────────────
1 │ 1 13 103 26 32
2 │ 2 13 101 26 33
3 │ 3 13 103 25 32
4 │ 2 11 101 26 32
5 │ 2 12 102 25 34
6 │ 1 11 102 25 34
7 │ 2 11 103 26 32
8 │ 3 13 102 25 32
9 │ 1 12 103 25 33
10 │ 3 11 101 26 32
julia> gdf = groupby(df, :id)
GroupedDataFrame with 3 groups based on key: id
First Group (3 rows): id = 1
Row │ id A B C D
│ Int64 Int64 Int64 Int64 Int64
─────┼───────────────────────────────────
1 │ 1 13 103 26 32
2 │ 1 11 102 25 34
3 │ 1 12 103 25 33
⋮
Last Group (3 rows): id = 3
Row │ id A B C D
│ Int64 Int64 Int64 Int64 Int64
─────┼───────────────────────────────────
1 │ 3 13 103 25 32
2 │ 3 13 102 25 32
3 │ 3 11 101 26 32
julia> function funnynt(id,Cols...; p1=1,p2=2,p3=3)
m= Matrix{Float64}(undef, length(id),length(Cols))
for i in eachindex(Cols)
if p3>0
m[:,i]= (id .+p1).*(Cols[i].+p2*p3)
else
m[:,i]= (id .+p1).*(Cols[i].+p2)
end
end
#ncols=Symbol.("id_c",1:length(Cols))
ncols=Symbol.("id_",names(df,Not(:id)))
return (;zip(ncols,eachcol(m))...)
end
funnynt (generic function with 1 method)
julia> transform(gdf,Cols(:) => funnynt=>AsTable)
10×9 DataFrame
Row │ id A B C D id_A id_B id_C id_D ⋯
│ Int64 Int64 Int64 Int64 Int64 Float64 Float64 Float64 Floa ⋯
─────┼─────────────────────────────────────────────────────────────────────
1 │ 1 13 103 26 32 38.0 218.0 64.0 7 ⋯
2 │ 2 13 101 26 33 57.0 321.0 96.0 11
3 │ 3 13 103 25 32 76.0 436.0 124.0 15
4 │ 2 11 101 26 32 51.0 321.0 96.0 11
5 │ 2 12 102 25 34 54.0 324.0 93.0 12 ⋯
6 │ 1 11 102 25 34 34.0 216.0 62.0 8
7 │ 2 11 103 26 32 51.0 327.0 96.0 11
8 │ 3 13 102 25 32 76.0 432.0 124.0 15
9 │ 1 12 103 25 33 36.0 218.0 62.0 7 ⋯
10 │ 3 11 101 26 32 68.0 428.0 128.0 15
1 column omitted
julia> transform(gdf,Cols(:) =>((x...)->funnynt(x...,p1=0;p3=0))=>AsTable)
10×9 DataFrame
Row │ id A B C D id_A id_B id_C id_D ⋯
│ Int64 Int64 Int64 Int64 Int64 Float64 Float64 Float64 Floa ⋯
─────┼─────────────────────────────────────────────────────────────────────
1 │ 1 13 103 26 32 15.0 105.0 28.0 3 ⋯
2 │ 2 13 101 26 33 30.0 206.0 56.0 7
3 │ 3 13 103 25 32 45.0 315.0 81.0 10
4 │ 2 11 101 26 32 26.0 206.0 56.0 6
5 │ 2 12 102 25 34 28.0 208.0 54.0 7 ⋯
6 │ 1 11 102 25 34 13.0 104.0 27.0 3
7 │ 2 11 103 26 32 26.0 210.0 56.0 6
8 │ 3 13 102 25 32 45.0 312.0 81.0 10
9 │ 1 12 103 25 33 14.0 105.0 27.0 3 ⋯
10 │ 3 11 101 26 32 39.0 309.0 84.0 10
1 column omitted
```
As for
> [@phantom](#):
>
> how the `zip` and `...` commands are working here
I don’t know if I’m able to clarify what actually happens behind the scene.
I go a bit for analogies in my experience with Julia (I don’t read much documentation, I must confess).
```julia
nt=(f1=1,f2=2,f3=3) # basic definition of a named tuple
nt=(;(f1=1,f2=2,f3=3)...) # the function of the splatting operator is to "open" the collection
# If you need to define a namedtuple dynamically, you need to follow these steps
fields=Symbol.('f',1:3) # a sequence of symbols
# and
values=[1,2,3] # a sequence of values
# below various ways to input it to the syntax (;kwargs...)
# which should be treated by the parser (?) in the same way as non-positional parameters of functions!?
# But here perhaps it is better for someone who knows about these things to be more specific.
(;zip(fields,values)...)
(;Pair.(fields,values)...)
(;tuple.(fields,values)...)
```
---
### Author:  [@sijo](https://discourse.julialang.org/u/sijo)
#### Post date: [December 23, 2022, 5:32pm UTC](https://discourse.julialang.org/t/run-multiple-instances-of-transform-on-specific-column-combinations-of-a-groupeddataframe-in-dataframes-mini-language/91736/23 "2022-12-23T17:32:03Z")
To expand a bit on @rocco_sprmnt21’s answer:
> [@phantom](#):
>
> could you explain the use of the `;` in the return statement of both examples of `fun`? e.g.
>
> ```julia
> return (; E, F, G)
>
> ```
>
> and in the second example.
>
> ```julia
> return (;
> Symbol("E_$suffix") => E,
> Symbol("F_$suffix") => F,
> Symbol("G_$suffix") => G,
> )
>
> ```
It’s a syntax for declaring named tuples. It’s not very well documented so I’ve submitted a [PR](https://github.com/JuliaLang/julia/pull/47981).
Basically, you can write `(; :a => 1, :b => 2)` to create the named tuple as if you had written `(a=1, b=2)`. The first form is more flexible: you can generate the name programmatically (as done above with `Symbol("E_$suffix")`), and if you want a tuple field to have the same name and same value as a variable, you don’t have to repeat yourself: `(; a=a)` can be shortened to `(; a)`. I used this above in `(; E, F, G)`.
> [@phantom](#):
>
> When I add the splat `...` operator after the `Cols` argument in the function declaration does that turn that argument in the function into an array?
It’s easy to try yourself to see what Julia does, for example:
```julia
julia> f(x...) = x;
julia> f(1,2,"text")
(1, 2, "text")
```
so if a function is declared with `x`, in the body the value of `x` is a tuple with all the arguments passed to the function (note that for keyword arguments it’s different, you get a special object that also holds the argument names).
The description above is for the case where `...` is used in the function declaration, as in your `funnynt(id, Cols...; p1=1, p2=2, p3=3)`. In the case of `(; zip(ncols,eachcol(m))...)` it’s different: here the splatting operator is not in a declaration, it’s more like a call. See the [FAQ](https://docs.julialang.org/en/v1/manual/faq/#What-does-the-...-operator-do?).
[Previous page](https://discourse.julialang.org/t/run-multiple-instances-of-transform-on-specific-column-combinations-of-a-groupeddataframe-in-dataframes-mini-language/91736.md?page=1)