# Nested map

**URL:** <https://discourse.julialang.org/t/nested-map/55889>\
**Category:** New to Julia\
**Created:** [February 23, 2021, 9:43pm UTC](https://discourse.julialang.org/t/nested-map/55889 "2021-02-23T21:43:11Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![eschoute](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/eschoute/32/8298_2.png) [@eschoute](https://discourse.julialang.org/u/eschoute)\
**Post date:** [February 23, 2021, 9:43pm UTC](https://discourse.julialang.org/t/nested-map/55889/1 "2021-02-23T21:43:11Z")

</div>

I’d like to apply a function to each element of a nested array. To illustrate

```julia
julia> arr = [[1,2,3],[4,5,6]]
2-element Array{Array{Int64,1},1}:
 [1, 2, 3]
 [4, 5, 6]

julia> map(arr) do ls
           map(ls) do x
               x+1
           end
       end
2-element Array{Array{Int64,1},1}:
 [2, 3, 4]
 [5, 6, 7]

```

Now say arr is nested 3 levels, then things start to get quite unwieldy. I thought of doing something like `map(k for i in arr for j in i for k in j)` but that flattens the list. Unfortunately, the syntax `map(k for i in arr, j in i, k in j)` doesn’t work like it does for nested arrays of fixed size. Any suggestion on how to implement an operation like this in a nicer way? Loops solve it somewhat nicely, but you are forced to iteratively push into some array.

---

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [February 23, 2021, 10:01pm UTC](https://discourse.julialang.org/t/nested-map/55889/2 "2021-02-23T22:01:12Z")

</div>

Here’s one solution:

```julia
nest_map(f, v::Array) = map(f, v);
nest_map(f, v::Array{<:Array}) = map(vi -> nest_map(f, vi), v);

```

```julia
julia> arr = [[[1,2], [3, 4, 5]], [[6,],[7,8,9,10]]];

julia> nest_map(x -> x + 1, arr)
2-element Vector{Vector{Vector{Int64}}}:
 [[2, 3], [4, 5, 6]]
 [[7], [8, 9, 10, 11]]

```

However, I should note that having arrays of arrays is a bit of a warning sign in julia. If those nested arrays are of the same size, chances are you would be better served by using a multidimensional array instead. E.g.

```julia
[i + (j - 1)*2 + (k-1)*4 for i in 1:2, j in 1:2, k in 1:2]
2×2×2 Array{Int64, 3}:
[:, :, 1] =
 1 3
 2 4

[:, :, 2] =
 5 7
 6 8

```

should be preferred over

```julia
[[[i + (j - 1)*2 + (k-1)*4 for i in 1:2] for j in 1:2] for k in 1:2]
2-element Vector{Vector{Vector{Int64}}}:
 [[1, 2], [3, 4]]
 [[5, 6], [7, 8]]

```

---

<div class="post-metadata">

**Author:** ![eschoute](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/eschoute/32/8298_2.png) [@eschoute](https://discourse.julialang.org/u/eschoute)\
**Post date:** [February 24, 2021, 2:01am UTC](https://discourse.julialang.org/t/nested-map/55889/3 "2021-02-24T02:01:25Z")

</div>

The nested arrays are not of the same size so I’m stuck with this kind of solution, unfortunately. I guess it can’t be much nicer than nestedd map blocks.

---

<div class="post-metadata">

**Author:** ![Skoffer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/skoffer/32/378_2.png) [@Skoffer](https://discourse.julialang.org/u/Skoffer)\
**Post date:** [February 24, 2021, 12:10pm UTC](https://discourse.julialang.org/t/nested-map/55889/5 "2021-02-24T12:10:47Z")

</div>

For triple nested array you can use the following

```julia
arr_arr_arr .|> x -> x .|> x -> x .|> f 

```

As for recursive, it’s not very good to use `isa`, especially with nonconrete types. It’s better to use multiple dispatch (not sure, maybe something like that already exists in `Base`)

```julia
apply_function(f, x) = f(x)
apply_function(f, x::AbstractArray) = apply_function.(f, x)

```

---

<div class="post-metadata">

**Author:** ![sijo](https://avatars.discourse-cdn.com/v4/letter/s/da6949/32.png) [@sijo](https://discourse.julialang.org/u/sijo)\
**Post date:** [February 24, 2021, 1:23pm UTC](https://discourse.julialang.org/t/nested-map/55889/6 "2021-02-24T13:23:10Z")

</div>

Say you have

```julia
arr = [[[1,2], [3, 4, 5]], [[6,],[7,8,9,10]]]
myf(x) = x+0.1

```

The following generalizes well to n levels of nesting (just replace 3 with the appropriate value):

```julia
julia> ∘(fill(f->x->map(f, x), 3)...)(myf)(arr)
2-element Vector{Vector{Vector{Float64}}}:
 [[1.1, 2.1], [3.1, 4.1, 5.1]]
 [[6.1], [7.1, 8.1, 9.1, 10.1]]

```

Hard to read so I would not use it 😉

It’s interesting that with a [curried map](https://github.com/JuliaLang/julia/issues/35293) we could just write `∘(fill(map, 3)...)(myf)(arr)`!

This makes me realize that the question in this thread is a good example why a curried map would be useful. For the special case of 3-level nesting you could just write:

```julia
map(map(map(myf)))(arr)

```

or

```julia
(myf |> map |> map |> map)(arr)

```

or

```julia
arr |> (map∘map∘map)(myf)

```

Proof of concept:

```julia
arr = [[[1,2], [3, 4, 5]], [[6,],[7,8,9,10]]]
myf(x) = x+0.1

# Curried version of map
mapper(f) = x->map(f, x)

arr |> (mapper∘mapper∘mapper)(myf)

# Output:
2-element Vector{Vector{Vector{Float64}}}:
 [[1.1, 2.1], [3.1, 4.1, 5.1]]
 [[6.1], [7.1, 8.1, 9.1, 10.1]]

```

---

<div class="post-metadata">

**Author:** ![eschoute](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/eschoute/32/8298_2.png) [@eschoute](https://discourse.julialang.org/u/eschoute)\
**Post date:** [February 24, 2021, 2:36pm UTC](https://discourse.julialang.org/t/nested-map/55889/7 "2021-02-24T14:36:51Z")

</div>

I like how you use composition of map functions. When I think about what I would expect Julia to look like, it would use some sort of broadcasting. Indeed if the nested array was some k-dimensional array instead of nested k levels, then a simple `f.(arr)` would suffice. Is there a way to specify the dimension along which is broadcast? Or to only broadcast at the leaves?

An alternative is to attempt to recreate an imperative loop.

```julia
[[foo.(y) for y in x] for x in arr]

```

But turning this into a loop requires a lot of, say, `push!` calls to iteratively build an array.

```julia
julia> vec = Vector{Vector{Vector{Int}}}()
Array{Array{Int64,1},1}[]

julia> for x in arr
           vecx = Vector{Vector{Int}}()
           for y in x
               vecy = Int[]
               for z in y
                   push!(vecy, foo(z))
               end
               push!(vecx, vecy)
           end
           push!(vec, vecx)
       end

julia> vec
2-element Array{Array{Array{Int64,1},1},1}:
 [[2, 3], [4, 5, 6]]
 [[7, 9]]

```

Loops don’t happen to be expressions as well right (like Scala)? I think in Scala you can write

```julia
res = for x in arr
    for y in x
        for z in y
            foo(z)
        end
    end
end

```

---

<div class="post-metadata">

**Author:** ![Henrique\_Becker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/henrique_becker/32/15443_2.png) [@Henrique\_Becker](https://discourse.julialang.org/u/Henrique_Becker)\
**Post date:** [February 24, 2021, 2:49pm UTC](https://discourse.julialang.org/t/nested-map/55889/8 "2021-02-24T14:49:53Z")

</div>

What exactly is the problem with the `nested_map` from @mason? The only case I see in which it falls short is if you want to maps specifically `k` levels and your nested structure has more than `k` levels (in this case `nested_map` has no notion of a threshold/max\_depth in which it should stop, it will keep recursively applying until the elements are not `AbstractVector`s anymore).

---

<div class="post-metadata">

**Author:** ![eschoute](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/eschoute/32/8298_2.png) [@eschoute](https://discourse.julialang.org/u/eschoute)\
**Post date:** [February 24, 2021, 2:54pm UTC](https://discourse.julialang.org/t/nested-map/55889/9 "2021-02-24T14:54:57Z")

</div>

Ah, I didn’t realize that aspect of `nest_map`, where it uses dispatch to create a base case on the type. Yes, that would address it nicely.

---

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [February 26, 2021, 11:09pm UTC](https://discourse.julialang.org/t/nested-map/55889/10 "2021-02-26T23:09:27Z")

</div>

Okay, so I was thinking today about how it’s kinda annoying you can’t do recursion inside a `do` block, but then I realized you totally can!

```julia
julia> map([[[1,2], [3, 4, 5]], [[6,],[7,8,9,10]]]) do x
           if x isa Array
               map(var"#self#", x)
           else
               x + 1
           end
       end
2-element Vector{Vector{Vector{Int64}}}:
 [[2, 3], [4, 5, 6]]
 [[7], [8, 9, 10, 11]]

```

I wouldn’t really recommend doing this, but it does work (assuming you’re on version 1.3 or newer where the `var` string macro was introduced). Just thought it was neat.

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [February 26, 2021, 11:23pm UTC](https://discourse.julialang.org/t/nested-map/55889/11 "2021-02-26T23:23:43Z")

</div>

Please, _please_ don’t do it in the production code (but I love the snippet 😂).

---

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [February 27, 2021, 12:06am UTC](https://discourse.julialang.org/t/nested-map/55889/12 "2021-02-27T00:06:42Z")

</div>

In case anyone is wondering why Takafumi thinks this is such a bad idea: there are various language constructs which might create closures, so writing code that depends on which function in a closure it’s written in is a _bad_ idea. For instance, for current versions of julia, we have that this:

```julia
julia> map([[[1,2], [3, 4, 5]], [[6,],[7,8,9,10]]]) do x
           if x isa Array
               map(var"#self#", x)
           else
               x + 1
           end
       end

```

is equivalent to this:

```julia
julia> map([[[1,2], [3, 4, 5]], [[6,],[7,8,9,10]]]) do x
           if x isa Array
               [var"#self#"(y) for y in x]
           else
               x + 1
           end
       end

```

but it won’t work on version 1.7 onwards due to [lowering: remove incorrect comprehension eta reduction by vtjnash · Pull Request #39139 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/pull/39139)!

There are various other contexts where using `self` would be a bad idea too, like inside `@async ` or `Threads.@spawn` and so on.

---

<div class="post-metadata">

**Author:** ![sijo](https://avatars.discourse-cdn.com/v4/letter/s/da6949/32.png) [@sijo](https://discourse.julialang.org/u/sijo)\
**Post date:** [February 27, 2021, 8:56am UTC](https://discourse.julialang.org/t/nested-map/55889/13 "2021-02-27T08:56:56Z")

</div>

What if we do something like this:

```julia
map([[[1,2], [3, 4, 5]], [[6,],[7,8,9,10]]]) do x
    self = var"#self#"
    if x isa Array
        [self(y) for y in x]
    else
        x + 1
    end
end

```

Is this also risky? It doesn’t look different from other uses of recursion…

---

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [February 27, 2021, 7:01pm UTC](https://discourse.julialang.org/t/nested-map/55889/14 "2021-02-27T19:01:52Z")

</div>

Yes,that will work, but the point is that it’s easy to get wrong, and makes the meaning of code highly dependant on what’s enclosing it. For instance,

```julia
f(x)

```

should really always be equivalent to

```julia
@sync @async f(x)

```

but here, if we wrote

```julia
map([[[1,2], [3, 4, 5]], [[6,],[7,8,9,10]]]) do x
    @sync @async begin
        self = var"#self#"
        if x isa Array
            [self(y) for y in x]
        else
            x + 1
        end
    end
end

```

it’ll blow up in your face because. It’s always possible to use `self` safely and carefully, but the point is just that using it can subtly violate a lot of naive expectations. It’s bad style.