# Filtering keys out of named tuples

**URL:** <https://discourse.julialang.org/t/filtering-keys-out-of-named-tuples/73564>\
**Category:** Performance\
**Tags:** namedtuple\
**Created:** [December 23, 2021, 11:57pm UTC](https://discourse.julialang.org/t/filtering-keys-out-of-named-tuples/73564 "2021-12-23T23:57:08Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [December 23, 2021, 11:57pm UTC](https://discourse.julialang.org/t/filtering-keys-out-of-named-tuples/73564/1 "2021-12-23T23:57:08Z")

</div>

What’s the right way to filter a key out of a named tuple? Example:

```julia
nt = (;a = 1, b = 2)
filter(x->x[1] == :a,nt) # Errors
filter(x->x[1] == :a,Base.Pairs(nt,keys(nt))) # Allocates an entire dictionary

```

What’s the right way to do this and get another NamedTuple back out instead of going through a dictionary and losing all inference/performance? Does this have an issue in Base already?

---

<div class="post-metadata">

**Author:** ![Henrique\_Becker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/henrique_becker/32/15443_2.png) [@Henrique\_Becker](https://discourse.julialang.org/u/Henrique_Becker)\
**Post date:** [December 24, 2021, 12:30am UTC](https://discourse.julialang.org/t/filtering-keys-out-of-named-tuples/73564/2 "2021-12-24T00:30:18Z")

</div>

I do believe the following is faster, can you benchmark it for your use case and give some feedback?

```julia
julia> filter_nt_fields(f, nt) = NamedTuple{filter(f, keys(nt))}(nt)
filter_nt_fields (generic function with 1 method)

julia> nt = (;a = 1, b = 2, c = 3)
(a = 1, b = 2, c = 3)

julia> filter_nt_fields(!=(:b), nt)
(a = 1, c = 3)

```

Note, however, that this is an inherently type-unstable operation.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [December 24, 2021, 12:37am UTC](https://discourse.julialang.org/t/filtering-keys-out-of-named-tuples/73564/3 "2021-12-24T00:37:48Z")

</div>

> [@Henrique\_Becker](#):
>
> Note, however, that this is an inherently type-unstable operation.

Not with constant prop. You should be able with constant prop to know `takeout(nt,(:b,))` and know that it will be (:a,:c) left. We just don’t have a method for it 😅. So yeah, maybe filter isn’t the right solution but that means there’s a missing verb.

---

<div class="post-metadata">

**Author:** ![Henrique\_Becker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/henrique_becker/32/15443_2.png) [@Henrique\_Becker](https://discourse.julialang.org/u/Henrique_Becker)\
**Post date:** [December 24, 2021, 1:30am UTC](https://discourse.julialang.org/t/filtering-keys-out-of-named-tuples/73564/4 "2021-12-24T01:30:02Z")

</div>

The `keys` method for `NamedTuple` return a `Tuple` and the `filter` method for `Tuple` acts directly over the `Tuple` (do not `collect` in a `Vector`) and return a `Tuple`, so theoretically, there the potential for constant propagation all the way through, did you try to see if it happens in your use case?

---

<div class="post-metadata">

**Author:** ![mcabbott](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mcabbott/32/6603_2.png) [@mcabbott](https://discourse.julialang.org/u/mcabbott)\
**Post date:** [December 24, 2021, 2:48am UTC](https://discourse.julialang.org/t/filtering-keys-out-of-named-tuples/73564/5 "2021-12-24T02:48:20Z")

</div>

You can do `tail((; b=nothing, nt...,))`, which seems fast:

```julia
julia> nt = (a=1, b=2, c=3);

julia> @inline takeout(kill::Symbol, nt::NamedTuple) = Base.tail(merge(NamedTuple{(kill,)}((nothing,)), nt));

julia> @btime takeout(:b, $nt)
  min 0.875 ns, mean 0.946 ns (0 allocations)
(a = 1, c = 3)

```

(This will break if `nt` doesn’t have `:b`, of course.)

---

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [December 24, 2021, 6:18am UTC](https://discourse.julialang.org/t/filtering-keys-out-of-named-tuples/73564/6 "2021-12-24T06:18:50Z")

</div>

Does this help, or you need more general filtering with an arbitrary predicate?

```julia
julia> nt = (;a = 1, b = 2)
(a = 1, b = 2)

julia> nt[(:a,)]
(a = 1,)

julia> Base.structdiff(nt, NamedTuple{(:a,)})
(b = 2,)

julia> using Accessors
julia> @delete nt.a
(b = 2,)

julia> using InvertedIndices
julia> nt[Not(:a)] # if/when https://github.com/JuliaData/InvertedIndices.jl/pull/29 lands
(b = 2,)

```

---

<div class="post-metadata">

**Author:** ![rocco\_sprmnt21](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rocco_sprmnt21/32/20127_2.png) [@rocco\_sprmnt21](https://discourse.julialang.org/u/rocco_sprmnt21)\
**Post date:** [December 25, 2021, 9:58pm UTC](https://discourse.julialang.org/t/filtering-keys-out-of-named-tuples/73564/7 "2021-12-25T21:58:33Z")

</div>

```julia
julia> @btime nt_b=NamedTuple{filter(!=(:b), keys($nt))}($nt)
  0.001 ns (0 allocations: 0 bytes)
(a = 1, c = 3)

julia> @btime $nt[filter(!=(:b), keys($nt))]
  0.001 ns (0 allocations: 0 bytes)
(a = 1, c = 3)

```

---

<div class="post-metadata">

**Author:** ![JeffreySarnoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeffreysarnoff/32/1980_2.png) [@JeffreySarnoff](https://discourse.julialang.org/u/JeffreySarnoff)\
**Post date:** [December 25, 2021, 10:54pm UTC](https://discourse.julialang.org/t/filtering-keys-out-of-named-tuples/73564/8 "2021-12-25T22:54:14Z")

</div>

Much of the benchmarking above presumes you know ahead of time the one symbol that want to remove, and you always want to remove that same symbol. The timings and allocations increase when both the named tuple and the symbol are not fixed. You may want to remove more than one key. Base uses `Base.structdiff` for this kind of manipulation. One approach:

```julia
drop(nt::NamedTuple, key::Symbol) = 
    Base.structdiff(nt, NamedTuple{(key,)})
drop(nt:: NamedTuple, keys::NTuple{N,Symbol}) where {N} =
    Base.structdiff(nt, NamedTuple{keys})

```

When you do know which symbol[s] are to be dropped from the keys, it is much more performant to work with function[s] that use that information immediately. When the same field[s] is/are removed in different parts of the source code, you may prefer to specialize the names.

```julia
drop_baz(nt) = Base.structdiff(nt, NamedTuple{(:baz,)})
drop_b_c(nt) = Base.structdiff(nt, NamedTuple{(:b, :c)})

```

Macroizing that process makes sense.

---

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [December 26, 2021, 6:51am UTC](https://discourse.julialang.org/t/filtering-keys-out-of-named-tuples/73564/9 "2021-12-26T06:51:43Z")

</div>

Performance of these variants is exactly the same under constant propagation, so there is no need to define functions separately for each key name:

```julia
julia> nt = (a=nothing, b=2, c='a')

julia> @btime Base.structdiff($nt, NamedTuple{(:a,)})
  1.589 ns (0 allocations: 0 bytes)
(b = 2, c = 'a')

# your drop() function, with @inline:
julia> @inline drop(nt::NamedTuple, key::Symbol) = 
           Base.structdiff(nt, NamedTuple{(key,)})
julia> @btime drop($nt, :a)
  1.587 ns (0 allocations: 0 bytes)
(b = 2, c = 'a')

# implementation in an existing package:
julia> using Accessors
julia> @btime @delete $nt.a
  1.586 ns (0 allocations: 0 bytes)
(b = 2, c = 'a')

```

---

<div class="post-metadata">

**Author:** ![JeffreySarnoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeffreysarnoff/32/1980_2.png) [@JeffreySarnoff](https://discourse.julialang.org/u/JeffreySarnoff)\
**Post date:** [December 26, 2021, 7:29pm UTC](https://discourse.julialang.org/t/filtering-keys-out-of-named-tuples/73564/10 "2021-12-26T19:29:05Z")

</div>

Yes. I was aware of the timing information; imo it is better practice to wrap uses of nonexported Base functions inside a user function, which may be inlined.

The multiple naming was to make clear the different specific intents, not to imply that need be done. I have modified the comment to clarify.