# The filter function is non-intuitive

**URL:** <https://discourse.julialang.org/t/the-filter-function-is-non-intuitive/45674>\
**Category:** Internals & Design\
**Created:** [August 28, 2020, 10:46am UTC](https://discourse.julialang.org/t/the-filter-function-is-non-intuitive/45674 "2020-08-28T10:46:47Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![TheLateKronos](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/thelatekronos/32/12824_2.png) [@TheLateKronos](https://discourse.julialang.org/u/TheLateKronos)\
**Post date:** [August 28, 2020, 10:46am UTC](https://discourse.julialang.org/t/the-filter-function-is-non-intuitive/45674/1 "2020-08-28T10:46:47Z")

</div>

So this is the current behaviour of the filter function, when given a function that returns a boolean and a collection (taken from the documentation):

```julia
julia> a = 1:10
1:10

julia> filter(isodd, a)
5-element Array{Int64,1}:
 1
 3
 5
 7
 9

```

I would argue that this is counter-inntuitive, at least to me, and I would like to discuss why too see what others think.

So my problem with it is the fact that a filter removes, ~~by definition~~ in my intuition. Something is filtered out from the whole. So to my mind, the function-call `filter(isodd, a)` is a filter that applies to the elements of `a` that are odd. This means that when the odd numbers are filtered, the function should, in my mind, return the even numbers.

Am I alone in this opinion? And as it is a breaking change, it is even a point to discuss this? I am ~~thinking it could be changed for 2.0, and that it is therefore a worthwhile discussion~~ interested in hearing other opinions on this.

---

<div class="post-metadata">

**Author:** ![GunnarFarneback](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gunnarfarneback/32/1827_2.png) [@GunnarFarneback](https://discourse.julialang.org/u/GunnarFarneback)\
**Post date:** [August 28, 2020, 11:11am UTC](https://discourse.julialang.org/t/the-filter-function-is-non-intuitive/45674/2 "2020-08-28T11:11:11Z")

</div>

This change would be massively breaking and contrary to every other programming language with a filter function, but in no way improve the functional power of the function. No, this will not be considered.

---

<div class="post-metadata">

**Author:** ![pkofod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pkofod/32/2179_2.png) [@pkofod](https://discourse.julialang.org/u/pkofod)\
**Post date:** [August 28, 2020, 11:14am UTC](https://discourse.julialang.org/t/the-filter-function-is-non-intuitive/45674/3 "2020-08-28T11:14:36Z")

</div>

Sieve would probably be a better name but… yeah as stated above you’d have to really argue that it’s worth it 🙂

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [August 28, 2020, 11:19am UTC](https://discourse.julialang.org/t/the-filter-function-is-non-intuitive/45674/4 "2020-08-28T11:19:14Z")

</div>

> [@TheLateKronos](#):
>
> So my problem with it is the fact that a filter removes, by definition.

Not really. A filter usually _separates_ something into two parts. It is up to the user to decide what is kept (could be both, just needed to be separate). Eg an [aggregate grading sieve](https://en.wikipedia.org/wiki/Sieve_analysis) would keep _all_ parts, all of which are needed for the analysis.

The lesson is that one should not rely on intuition for these things — just read the docs if you are unsure.

> [@TheLateKronos](#):
>
> it is even a point to discuss this?

Probably not, see

> [@PSA: Julia is not at that stage of development anymore](https://discourse.julialang.org/t/psa-julia-is-not-at-that-stage-of-development-anymore/44872):
>
> You heard about this brand new exciting programming language called Julia and have some suggestions for how it should work. Wonderful! We’re happy to have you here and help you learn the language and listen to your ideas! However, please be aware that “new” is a relative term. Yes, Julia is newer than, say, C, which was first created in 1972 and standardized in 1989. It is not, however, new in the sense that it is still being freely designed and changed in arbitrary ways. Julia development start…

---

<div class="post-metadata">

**Author:** ![pfitzseb](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pfitzseb/32/45566_2.png) [@pfitzseb](https://discourse.julialang.org/u/pfitzseb)\
**Post date:** [August 28, 2020, 11:32am UTC](https://discourse.julialang.org/t/the-filter-function-is-non-intuitive/45674/5 "2020-08-28T11:32:41Z")

</div>

Also, fwiw, all `filter` implementations I’ve ever seen work this way and return an array/iterable with elements for which the predicate returns `true`.

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [August 28, 2020, 11:33am UTC](https://discourse.julialang.org/t/the-filter-function-is-non-intuitive/45674/6 "2020-08-28T11:33:08Z")

</div>

This trips me up often as well because I think of the function filtering things out, but as others have said, there’s already a traditional meaning of this higher order function in other languages to consider. What might be more plausible is introducing a `reject` verb and maybe a corresponding `select`, although that already has many other meanings.

---

<div class="post-metadata">

**Author:** ![anon37204545](https://avatars.discourse-cdn.com/v4/letter/a/439d5e/32.png) [@anon37204545](https://discourse.julialang.org/u/anon37204545)\
**Post date:** [August 28, 2020, 12:21pm UTC](https://discourse.julialang.org/t/the-filter-function-is-non-intuitive/45674/7 "2020-08-28T12:21:01Z")

</div>

You can define something like this

```julia
filterout(f, a) = filter(!f, a)

```

to remedy your pain (from the phrase _filter something out_).

To me it makes more sense to apply a condition directly (i.e. the current `filter` approach), instead of negating it. Something akin to

```julia
if condition
    A
else
    B
end

```

instead of

```julia
if !condition
     B
else
     A
end

```

It simply requires less mental gymnastics.

---

<div class="post-metadata">

**Author:** ![xiaodai](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xiaodai/32/15937_2.png) [@xiaodai](https://discourse.julialang.org/u/xiaodai)\
**Post date:** [August 28, 2020, 1:01pm UTC](https://discourse.julialang.org/t/the-filter-function-is-non-intuitive/45674/8 "2020-08-28T13:01:23Z")

</div>

`where` is less ambiguous. `filter` was used by dplyr which uses odd verbs like `arrange`, `mutate`.

---

<div class="post-metadata">

**Author:** ![mbaz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mbaz/32/17295_2.png) [@mbaz](https://discourse.julialang.org/u/mbaz)\
**Post date:** [August 28, 2020, 1:36pm UTC](https://discourse.julialang.org/t/the-filter-function-is-non-intuitive/45674/9 "2020-08-28T13:36:40Z")

</div>

> [@Tamas\_Papp](#):
>
> > [@TheLateKronos](#):
> >
> > So my problem with it is the fact that a filter removes, by definition.
> 
> Not really. A filter usually _separates_ something into two parts. It is up to the user to decide what is kept (could be both, just needed to be separate).

Depends on the context. In signal processing, a filter rejects part of the signal – there’s no way to get it back. One says, for example, “I need to filter this interference” to mean “reject the interference”. You can use filters to decompose a signal into its components, but you need more than one filter (for example, in a sound equalizer).

Coming from an EE, not computing, background, the behavior of `filter` in programming languages is counter-intuitive to me too, but I’ve gotten used to it.

---

<div class="post-metadata">

**Author:** ![ianfiske](https://avatars.discourse-cdn.com/v4/letter/i/58f4c7/32.png) [@ianfiske](https://discourse.julialang.org/u/ianfiske)\
**Post date:** [August 28, 2020, 1:46pm UTC](https://discourse.julialang.org/t/the-filter-function-is-non-intuitive/45674/10 "2020-08-28T13:46:39Z")

</div>

Although I completely agree with the sentiment that this should _not_ be changed for all the reasons mentioned above, it’s worth noting that even in common use like “coffee filter”, not just EE use, “filter” often implies the removal. So confusion is warranted for any relatively new programmers, though they’d experience this regardless of language choice.

But it’s just something to get used to since, like others have said, this is the convention in all programming languages.

edit: also see [Filter (higher-order function) - Wikipedia](https://en.wikipedia.org/wiki/Filter_(higher-order_function))

---

<div class="post-metadata">

**Author:** ![non-Jedi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/non-jedi/32/3645_2.png) [@non-Jedi](https://discourse.julialang.org/u/non-Jedi)\
**Post date:** [August 28, 2020, 1:55pm UTC](https://discourse.julialang.org/t/the-filter-function-is-non-intuitive/45674/11 "2020-08-28T13:55:07Z")

</div>

For what it’s worth, in the domain of manufacturing filtration is an operation which separates a mixed liquid/solid stream into a liquid stream and solid stream. We call the liquid stream the “filtrate”, and the solids form a “cake”. _Usually_ the filtrate is the product you’re most interested in but not always. If we treat the `filter` function as an analogy to this kind of filtration, it might make sense to have a keyword argument that controls which stream is returned (this would also allow getting both if wanted). Something like `product=:filtrate`, `product=:cake`, or `product=:both`.

Personally, I don’t think this is very beneficial, but I thought I would throw it out there.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [August 28, 2020, 2:23pm UTC](https://discourse.julialang.org/t/the-filter-function-is-non-intuitive/45674/12 "2020-08-28T14:23:26Z")

</div>

> [@mbaz](#):
>
> In signal processing, a filter eliminates part of the signal – there’s no way to get it back.

Sure, there are lots of intuitive interpretations that go either way. The point is that there isn’t a single one, so people should just read the docs.

The only unambiguous approach I can imagine is to spell it out, cf [`COMMON-LISP:REMOVE-IF-NOT`](http://clhs.lisp.se/Body/f_rm_rm.htm) & friends.

The [Wikipedia page for filter](https://en.wikipedia.org/wiki/Filter_(higher-order_function)) has a nice table summarizing syntax in various languages. I guess that `filter` in particular comes from the ML family historically, but I am not sure. In any case, Julia’s usage seems to be the common one in programming.

---

<div class="post-metadata">

**Author:** ![Zach\_Christensen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/zach_christensen/32/7220_2.png) [@Zach\_Christensen](https://discourse.julialang.org/u/Zach_Christensen)\
**Post date:** [August 28, 2020, 2:32pm UTC](https://discourse.julialang.org/t/the-filter-function-is-non-intuitive/45674/13 "2020-08-28T14:32:58Z")

</div>

I have an idea. It might be a bit radical but I think it could solve this problem without any new functions or breaking changes.

```julia
filter(!isodd, 1:10)

```

😃

---

<div class="post-metadata">

**Author:** ![GunnarFarneback](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gunnarfarneback/32/1827_2.png) [@GunnarFarneback](https://discourse.julialang.org/u/GunnarFarneback)\
**Post date:** [August 28, 2020, 2:42pm UTC](https://discourse.julialang.org/t/the-filter-function-is-non-intuitive/45674/14 "2020-08-28T14:42:02Z")

</div>

> [@mbaz](#):
>
> In signal processing, a filter eliminates part of the signal – there’s no way to get it back.

And as it happens some of the most common filters like the lowpass filters specify by the name what is to be kept, just like programming languages do with their predicate.

---

<div class="post-metadata">

**Author:** ![TheLateKronos](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/thelatekronos/32/12824_2.png) [@TheLateKronos](https://discourse.julialang.org/u/TheLateKronos)\
**Post date:** [August 28, 2020, 3:16pm UTC](https://discourse.julialang.org/t/the-filter-function-is-non-intuitive/45674/15 "2020-08-28T15:16:19Z")

</div>

Thanks for enlightening replies. So it seems like I am not alone in my intuition, but that as it is massivly breaking and there is a lot of presidence for this implementation, things are fine for now.

If I keep having to think hard to have this make sense, I will implement my own `filterout` function, as suggested by @anon37204545. Thanks for everyone’s time and opinions 🙂

---

<div class="post-metadata">

**Author:** ![pauljurczak](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pauljurczak/32/921_2.png) [@pauljurczak](https://discourse.julialang.org/u/pauljurczak)\
**Post date:** [August 28, 2020, 3:34pm UTC](https://discourse.julialang.org/t/the-filter-function-is-non-intuitive/45674/16 "2020-08-28T15:34:39Z")

</div>

> [@StefanKarpinski](#):
>
> introducing a `reject` verb and maybe a corresponding `select`

How about `keep` and `reject`?

---

<div class="post-metadata">

**Author:** ![mbaz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mbaz/32/17295_2.png) [@mbaz](https://discourse.julialang.org/u/mbaz)\
**Post date:** [August 28, 2020, 3:49pm UTC](https://discourse.julialang.org/t/the-filter-function-is-non-intuitive/45674/17 "2020-08-28T15:49:05Z")

</div>

Indeed – and the filter’s output is “the filtered signal”. I guess that’s why I never had as much trouble with `filter` as defined in programming as the OP. The point I was trying to make is that, in many contexts, a filter does not “separate” something into two parts which you get to keep, which was @Tamas_Papp’s definition.

---

<div class="post-metadata">

**Author:** ![TheLateKronos](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/thelatekronos/32/12824_2.png) [@TheLateKronos](https://discourse.julialang.org/u/TheLateKronos)\
**Post date:** [August 28, 2020, 8:00pm UTC](https://discourse.julialang.org/t/the-filter-function-is-non-intuitive/45674/18 "2020-08-28T20:00:26Z")

</div>

That is a fine band-aid, and I have concidered it. The original problem was however that I currently feel the need to do mental gymnastics to understand the code I write well, and this just feels like adding a flip to those gymnastics… Then it is better to just accept the current way the function works IMO

---

<div class="post-metadata">

**Author:** ![TheLateKronos](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/thelatekronos/32/12824_2.png) [@TheLateKronos](https://discourse.julialang.org/u/TheLateKronos)\
**Post date:** [August 29, 2020, 8:55am UTC](https://discourse.julialang.org/t/the-filter-function-is-non-intuitive/45674/19 "2020-08-29T08:55:19Z")

</div>

Huge fan of this idea. Really like the lack of ambiguity.

I personally like keep/discard or reject/select, as they feel like natural conceptual pairs.

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [August 29, 2020, 11:30am UTC](https://discourse.julialang.org/t/the-filter-function-is-non-intuitive/45674/20 "2020-08-29T11:30:07Z")

</div>

A filter can remove, and a filter can keep. A dust filter removes dust, an air filter lets through air. A bandpass filter passes through some frequency bands. A bandstop filter removes them.

The specification depends on whether it’s easier to enumerate the removed or kept parts. Neither is more intuitive or obvious.

> [@ianfiske](#):
>
> it’s worth noting that even in common use like “coffee filter”, not just EE use, “filter” often implies the removal.

What does a coffee filter let through? Coffee! 😉

[Next page](https://discourse.julialang.org/t/the-filter-function-is-non-intuitive/45674.md?page=2)