DataFrames transform list comprehension

New to Julia
1

Is there a way to achieve the below using list comprehsion?

My goal is to create columns lag1 to lagN by doing [dosomething for i in 1:N]

df = DataFrame(:ret => rand(100))
transform(df,:ret => (col -> lag(col,1)) => :lag1, 
             :ret => (col -> lag(col,2)) => :lag2,
             :ret => (col -> lag(col,3)) => :lag3,)

All my combinations of list comprehensions yielded errors…

Thanks for your help!

2 
julia> using DataFrames, ShiftedArrays

julia> df = DataFrame(ret = rand(100));

julia> transform!(df, ["ret" => (col -> lag(col, i)) => "lag$i" for i ∈ 1:3])
100Γ—4 DataFrame
 Row β”‚ ret        lag1             lag2             lag3            
     β”‚ Float64    Float64?         Float64?         Float64?        
─────┼──────────────────────────────────────────────────────────────
   1 β”‚ 0.624376   missing          missing          missing         
   2 β”‚ 0.605845         0.624376   missing          missing         
   3 β”‚ 0.0379192        0.605845         0.624376   missing         
   4 β”‚ 0.759851         0.0379192        0.605845         0.624376
   5 β”‚ 0.32622          0.759851         0.0379192        0.605845
3

This is pretty much what I do, except I think my pattern is usually to splat a generator rather than make a vector. ie,

transform!(df, ("ret" => (col -> lag(col, i)) => "lag$i" for i ∈ 1:3)...)

Not sure it matters much.

1 Like
4 
julia> @btime transform!(df, ["ret" => (col -> lag(col, i)) => "lag$i" for i ∈ 1:3]);
  46.236 ΞΌs (246 allocations: 12.38 KiB)

julia> @btime transform!(df, ("ret" => (col -> lag(col, i)) => "lag$i" for i ∈ 1:3)...);
  46.171 ΞΌs (266 allocations: 13.22 KiB)

:man_shrugging:

But I suppose the results might be different depending on the number of transformations to generate and other things, although one would hope that in most practical situations the cost of the actual transform swamps the construction of the call…

1 Like
6

It should not matter much if you splat or not - other parts of the code are much more expensive. I typically pass a vector, as it is easier to understand what is going on than when using splatting.

2 Likes
7

Great, thanks for the swift support!