# Performance of select! on a DataFrame

**URL:** <https://discourse.julialang.org/t/performance-of-select-on-a-dataframe/68343>\
**Category:** Performance\
**Tags:** dataframes\
**Created:** [September 17, 2021, 4:02pm UTC](https://discourse.julialang.org/t/performance-of-select-on-a-dataframe/68343 "2021-09-17T16:02:21Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![CameronBieganek](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cameronbieganek/32/6915_2.png) [@CameronBieganek](https://discourse.julialang.org/u/CameronBieganek)\
**Post date:** [September 17, 2021, 4:02pm UTC](https://discourse.julialang.org/t/performance-of-select-on-a-dataframe/68343/1 "2021-09-17T16:02:21Z")

</div>

Naively I would expect that dropping columns (in place) from a `DataFrame` with `select!` would be a very cheap operation, on the order of nanoseconds. But I’m getting times in the microsecond range:

```julia
julia> using DataFrames, BenchmarkTools

julia> @benchmark select!(df, :a) evals=1 setup=(df = DataFrame(a=rand(10_000), b = rand(10_000)))
BenchmarkTools.Trial: 10000 samples with 1 evaluation.
 Range (min … max): 1.431 μs … 8.846 μs ┊ GC (min … max): 0.00% … 0.00%
 Time (median): 1.916 μs ┊ GC (median): 0.00%
 Time (mean ± σ): 1.965 μs ± 272.973 ns ┊ GC (mean ± σ): 0.00% ± 0.00%

                 ▂▄▆▆▆█▆▆▆▅▄▃▂▂▁                               
  ▂▁▂▂▂▂▂▂▂▂▃▄▅▆██████████████████▇▇▆▆▆▅▅▄▄▃▄▃▃▃▃▃▃▃▃▃▂▂▂▂▂▂▂ ▄
  1.43 μs Histogram: frequency by time 2.65 μs <

 Memory estimate: 1.78 KiB, allocs estimate: 24.

```

Am I doing something wrong here?

---

<div class="post-metadata">

**Author:** ![bkamins](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bkamins/32/208538_2.png) [@bkamins](https://discourse.julialang.org/u/bkamins)\
**Post date:** [September 17, 2021, 4:08pm UTC](https://discourse.julialang.org/t/performance-of-select-on-a-dataframe/68343/2 "2021-09-17T16:08:20Z")

</div>

You are not doing it wrong. The way `select!` is implemented now is that it creates a new `DataFrame` and then overwrites the old one.

What you ask for is doable as an optimization (i.e. we would need to check that the user does only simple column selection) but it was considered that microseconds are fast enough. Do you have any specific use case where you would need this?

Note though that even non-trivial selections should not be expected to be fast, e.g. `select!(df, r"x", Not(:y), Between(:y, :z))` is quite complex to process and will be more expensive than nanoseconds.

---

<div class="post-metadata">

**Author:** ![CameronBieganek](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cameronbieganek/32/6915_2.png) [@CameronBieganek](https://discourse.julialang.org/u/CameronBieganek)\
**Post date:** [September 17, 2021, 4:12pm UTC](https://discourse.julialang.org/t/performance-of-select-on-a-dataframe/68343/3 "2021-09-17T16:12:50Z")

</div>

> [@bkamins](#):
>
> it was considered that microseconds are fast enough. Do you have any specific use case where you would need this?

That’s true, microseconds is probably fast enough. I was just experimenting in the REPL. I don’t have a real use case where selecting columns is a performance critical operation. 🙂