# DataFrames transform list comprehension

**URL:** <https://discourse.julialang.org/t/dataframes-transform-list-comprehension/52519>\
**Category:** New to Julia\
**Created:** [December 28, 2020, 4:18pm UTC](https://discourse.julialang.org/t/dataframes-transform-list-comprehension/52519 "2020-12-28T16:18:38Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![ZettEff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/zetteff/32/31976_2.png) [@ZettEff](https://discourse.julialang.org/u/ZettEff)\
**Post date:** [December 28, 2020, 4:18pm UTC](https://discourse.julialang.org/t/dataframes-transform-list-comprehension/52519/1 "2020-12-28T16:18:38Z")

</div>

Is there a way to achieve the below using _list comprehsion_?

My goal is to create columns lag1 to lagN by doing `[dosomething for i in 1:N]`

```julia
df = DataFrame(:ret => rand(100))
transform(df,:ret => (col -> lag(col,1)) => :lag1, 
             :ret => (col -> lag(col,2)) => :lag2,
             :ret => (col -> lag(col,3)) => :lag3,)

```

All my combinations of list comprehensions yielded errors…

Thanks for your help!

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [December 28, 2020, 4:27pm UTC](https://discourse.julialang.org/t/dataframes-transform-list-comprehension/52519/2 "2020-12-28T16:27:39Z")

</div>

```julia
julia> using DataFrames, ShiftedArrays

julia> df = DataFrame(ret = rand(100));

julia> transform!(df, ["ret" => (col -> lag(col, i)) => "lag$i" for i ∈ 1:3])
100×4 DataFrame
 Row │ ret lag1 lag2 lag3            
     │ Float64 Float64? Float64? Float64?        
─────┼──────────────────────────────────────────────────────────────
   1 │ 0.624376 missing missing missing         
   2 │ 0.605845 0.624376 missing missing         
   3 │ 0.0379192 0.605845 0.624376 missing         
   4 │ 0.759851 0.0379192 0.605845 0.624376
   5 │ 0.32622 0.759851 0.0379192 0.605845

```

---

<div class="post-metadata">

**Author:** ![tbeason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tbeason/32/15898_2.png) [@tbeason](https://discourse.julialang.org/u/tbeason)\
**Post date:** [December 28, 2020, 4:59pm UTC](https://discourse.julialang.org/t/dataframes-transform-list-comprehension/52519/3 "2020-12-28T16:59:09Z")

</div>

This is pretty much what I do, except I think my pattern is usually to splat a generator rather than make a vector. ie,

```julia
transform!(df, ("ret" => (col -> lag(col, i)) => "lag$i" for i ∈ 1:3)...)

```

Not sure it matters much.

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [December 28, 2020, 5:16pm UTC](https://discourse.julialang.org/t/dataframes-transform-list-comprehension/52519/4 "2020-12-28T17:16:38Z")

</div>

```julia
julia> @btime transform!(df, ["ret" => (col -> lag(col, i)) => "lag$i" for i ∈ 1:3]);
  46.236 μs (246 allocations: 12.38 KiB)

julia> @btime transform!(df, ("ret" => (col -> lag(col, i)) => "lag$i" for i ∈ 1:3)...);
  46.171 μs (266 allocations: 13.22 KiB)

```

🤷‍♂️

But I suppose the results might be different depending on the number of transformations to generate and other things, although one would hope that in most practical situations the cost of the actual transform swamps the construction of the call…

---

<div class="post-metadata">

**Author:** ![bkamins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bkamins/32/208538_2.png) [@bkamins](https://discourse.julialang.org/u/bkamins)\
**Post date:** [December 28, 2020, 6:20pm UTC](https://discourse.julialang.org/t/dataframes-transform-list-comprehension/52519/6 "2020-12-28T18:20:04Z")

</div>

It should not matter much if you splat or not - other parts of the code are much more expensive. I typically pass a vector, as it is easier to understand what is going on than when using splatting.

---

<div class="post-metadata">

**Author:** ![ZettEff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/zetteff/32/31976_2.png) [@ZettEff](https://discourse.julialang.org/u/ZettEff)\
**Post date:** [December 28, 2020, 8:33pm UTC](https://discourse.julialang.org/t/dataframes-transform-list-comprehension/52519/7 "2020-12-28T20:33:19Z")

</div>

Great, thanks for the swift support!