# Fastest way to create new column in DataFrames.jl

**URL:** <https://discourse.julialang.org/t/fastest-way-to-create-new-column-in-dataframes-jl/45933>\
**Category:** New to Julia\
**Created:** [September 2, 2020, 6:21am UTC](https://discourse.julialang.org/t/fastest-way-to-create-new-column-in-dataframes-jl/45933 "2020-09-02T06:21:48Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Post date:** [September 2, 2020, 6:21am UTC](https://discourse.julialang.org/t/fastest-way-to-create-new-column-in-dataframes-jl/45933/1 "2020-09-02T06:21:48Z")

</div>

I found 3 ways to create new columns in DataFrames.jl (two of which are equivalent)

```julia
df = DataFrame(a = rand(1_000_000), b= rand(1_000_000))

@time transform!(df, [:a, :b] => (a,b)->a + b)
@time transform!(df, [:a, :b] => (a,b)->a + b)

@time select!(df, :, [:a, :b] => (a,b)->a + b)
@time select!(df, :, [:a, :b] => (a,b)->a + b) # 0.1s

@time df[!, :c] = df[!, :a] .+ df[!, :b] #0.001s
@time df[!, :c] = df[!, :a] .+ df[!, :b]

```

The last method is much faster using `@time`. But it is not true when using `@benchmark`.

Why is `@time` much worse? When doing data manipulation I don’t tend to put these into smaller functions like this

```julia
sumab_df(df) = transform!(df, [:a, :b] => (a,b)->a + b)

@time sumab_df(df)
@time sumab_df(df)

```

So I tried to create a function `meh` to see why top 2 are slower. And I found this advice for performance

1. Using `df[!, :c] = fn(df[!, :a], df[!, :b)` where possible so you can get performance without putting operations in functions
2. Use named function rather than anonymous function where possible.

Are these advice on point?

> **Benchmarking code**
>
> ```julia
> 
> @benchmark transform!($df, [:a, :b] => (a,b)->a + b)
> # BenchmarkTools.Trial:
> # memory estimate: 7.63 MiB
> # allocs estimate: 97
> # --------------
> # minimum time: 1.638 ms (0.00% GC)
> # median time: 1.835 ms (0.00% GC)
> # mean time: 2.429 ms (16.38% GC)
> # maximum time: 11.841 ms (48.65% GC)
> # --------------
> # samples: 2058
> # evals/sample: 1
> @benchmark select!($df, :, [:a, :b] => (a,b)->a + b)
> # BenchmarkTools.Trial: 
> # memory estimate: 7.63 MiB
> # allocs estimate: 98
> # --------------
> # minimum time: 1.619 ms (0.00% GC)
> # median time: 1.759 ms (0.00% GC)
> # mean time: 2.316 ms (16.70% GC)
> # maximum time: 11.561 ms (48.30% GC)
> # --------------
> # samples: 2160
> # evals/sample: 1
> @benchmark select!($df, :, [:a, :b] => (a,b)->a .+ b)
> # BenchmarkTools.Trial: 
> # memory estimate: 7.63 MiB
> # allocs estimate: 98
> # --------------
> # minimum time: 1.642 ms (0.00% GC)
> # median time: 1.760 ms (0.00% GC)
> # mean time: 2.321 ms (16.66% GC)
> # maximum time: 11.834 ms (40.55% GC)
> # --------------
> # samples: 2154
> # evals/sample: 1
> 
> @benchmark df[!, :c] = df[!, :a] .+ df[!, :b]
> BenchmarkTools.Trial: 
> memory estimate: 7.63 MiB
> allocs estimate: 5
> --------------
> minimum time: 1.605 ms (0.00% GC)
> median time: 1.731 ms (0.00% GC)
> mean time: 2.291 ms (16.50% GC)
> maximum time: 11.518 ms (48.88% GC)
> --------------
> samples: 2180
> evals/sample: 1
> # Another way
> function meh(df, (cols, fn), outcol)
> cols_vals = getindex.(Ref(df), !, cols)
> df[!, outcol] = fn.(cols_vals...)
> df
> end
> 
> # passing in anonymouse function is slower
> @benchmark meh(df, ([:a, :b] => (a, b)-> a+b), :c)
> 
> sumab(a, b) = a + b
> # passing in named function is faster
> @benchmark meh(df, ([:a, :b] => sumab), :c)
> 
> @benchmark df[!, :c] = sumab.(df[!, :a], df[!, :b])
> 
> ```