# Should there be a \`mapfilter\` function or macro?

**URL:** <https://discourse.julialang.org/t/should-there-be-a-mapfilter-function-or-macro/65933>\
**Category:** General Usage\
**Created:** [August 6, 2021, 9:08am UTC](https://discourse.julialang.org/t/should-there-be-a-mapfilter-function-or-macro/65933 "2021-08-06T09:08:17Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![jules](https://avatars.discourse-cdn.com/v4/letter/j/41988e/32.png) [@jules](https://discourse.julialang.org/u/jules)\
**Post date:** [August 6, 2021, 9:08am UTC](https://discourse.julialang.org/t/should-there-be-a-mapfilter-function-or-macro/65933/1 "2021-08-06T09:08:17Z")

</div>

I often encounter a scenario, where I use `map` but then notice in the body of the closure that I want to throw away some values. I think there’s no simple way to do this. Here’s a mockup version:

```julia
itr = 1:100
mapfilter(itr) do i
    intermediate_result = first_function(i)
    if some_condition(intermediate_result)
        return second_function(intermediate_result)
    else
        return # result is filtered out
    end
end

```

The problem with list comprehensions is that the if condition can’t make use of intermediary values, so I would have to call `first_function` twice:

```julia
[second_function(first_function(i)) for i in itr if some_condition(first_function(i))]

```

A chained `filter` and `Iterators.map` doesn’t work because the state within `map` is not accessible to `filter`.

What would be the best way to get a function like this, preferrably with Base methods?

---

<div class="post-metadata">

**Author:** ![oxinabox](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oxinabox/32/206603_2.png) [@oxinabox](https://discourse.julialang.org/u/oxinabox)\
**Post date:** [August 6, 2021, 9:20am UTC](https://discourse.julialang.org/t/should-there-be-a-mapfilter-function-or-macro/65933/2 "2021-08-06T09:20:41Z")

</div>

A for loop containimg a call to `push!` is flexible, clear, and fast.  
You can `sizehint!` the vector you are pushing into up to maximum size it could be before you start.

---

<div class="post-metadata">

**Author:** ![jules](https://avatars.discourse-cdn.com/v4/letter/j/41988e/32.png) [@jules](https://discourse.julialang.org/u/jules)\
**Post date:** [August 6, 2021, 9:24am UTC](https://discourse.julialang.org/t/should-there-be-a-mapfilter-function-or-macro/65933/3 "2021-08-06T09:24:16Z")

</div>

That is true as long as it’s easy enough to specify the return type of the vector without running your function. It might be just `Int` but it might be `Horrible{Type{With{Many{Parameters}}}}`. Using `map` or list comprehensions thankfully spares me from doing that.

Taking just the first value to get the type wouldn’t work in type unstable scenarios, even if those should generally be avoided of course.

---

<div class="post-metadata">

**Author:** ![Skoffer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/skoffer/32/378_2.png) [@Skoffer](https://discourse.julialang.org/u/Skoffer)\
**Post date:** [August 6, 2021, 9:47am UTC](https://discourse.julialang.org/t/should-there-be-a-mapfilter-function-or-macro/65933/4 "2021-08-06T09:47:18Z")

</div>

You can do it by using `iterate` protocol with something like this:

```julia
struct MapFilter{F1, F2, T}
    f::F1
    cond::F2
    x::T
end

mapfilter(f, cond, x) = MapFilter(f, cond, x)

function Base.iterate(x::MapFilter, state = iterate(x.x))
    while true
        state === nothing && return nothing
        val, id = state
        state = iterate(x.x, id)
        y = x.f(val)
        x.cond(y) && return (y, state)
    end
end

function _collect(mf, state, out)
    for x in Iterators.rest(mf, state)
        push!(out, x)
    end

    return out
end

function Base.collect(mf::MapFilter)
    peel = iterate(mf)
    peel === nothing && return nothing
    val, state = peel
    out = [val]
    _collect(mf, state, out)
end

```

```julia
julia> collect(mapfilter(x -> x^2, x -> x < 10, [1, 2, 3, 4, 3, 5, 1]))
5-element Vector{Int64}:
 1
 4
 9
 9
 1

julia> collect(mapfilter(x -> x^2, x -> x < 10, [4, 5]))

```

With that said, I highly recommend to use packages which already have this (and more) functionality, like for example [GitHub - JuliaFolds/Transducers.jl: Efficient transducers for Julia](https://github.com/JuliaFolds/Transducers.jl)

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [August 6, 2021, 9:58am UTC](https://discourse.julialang.org/t/should-there-be-a-mapfilter-function-or-macro/65933/5 "2021-08-06T09:58:50Z")

</div>

The issue is (I think) that a priori the type of the collection can’t be known because it may depend on the filter function. For example, a `Vector{Union{T, Missing}}` may, after your mapfilter, collapse into a `Vector{T}` if your filter is `ismissing`. Unless mapfilter is specialised on that, it can’t infer the return type correctly (or has to guess and widen, like `filter` or `map` does currently I think).

---

<div class="post-metadata">

**Author:** ![sijo](https://avatars.discourse-cdn.com/v4/letter/s/da6949/32.png) [@sijo](https://discourse.julialang.org/u/sijo)\
**Post date:** [August 6, 2021, 10:18am UTC](https://discourse.julialang.org/t/should-there-be-a-mapfilter-function-or-macro/65933/6 "2021-08-06T10:18:58Z")

</div>

> [@jules](#):
>
> The problem with list comprehensions is that the if condition can’t make use of intermediary values, so I would have to call `first_function` twice:

Not sure if it’s a good idea but you can do e.g.

```julia
julia> [temp^2 for i in 1:100 for temp in (sqrt(i),) if temp < 2]
3-element Vector{Float64}:
 1.0
 2.0000000000000004
 2.9999999999999996

```

(For numbers the inner loop could be simplified as in `for temp = sqrt(i)` but I think `for temp in (sqrt(i,))` is more explicit about what’s going on.)