# Custom flatten version is faster than \`flatten()\`

**URL:** <https://discourse.julialang.org/t/custom-flatten-version-is-faster-than-flatten/16048>\
**Category:** General Usage\
**Created:** [October 8, 2018, 11:58pm UTC](https://discourse.julialang.org/t/custom-flatten-version-is-faster-than-flatten/16048 "2018-10-08T23:58:51Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![affans](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/affans/32/11911_2.png) [@affans](https://discourse.julialang.org/u/affans)\
**Post date:** [October 8, 2018, 11:58pm UTC](https://discourse.julialang.org/t/custom-flatten-version-is-faster-than-flatten/16048/1 "2018-10-08T23:58:51Z")

</div>

Consider an array of matrices i.e. `a = [rand(3, 3) for i = 1:1000]`. I have two ways to “flatten” this array:

```julia
function naive_flatten(a)
    ret = Array{Float64}(undef, 0)
    @inbounds for i=1:length(a) 
        append!(ret, vec(a[i]))
    end
end

using BenchmarkTools; using Base.Iterators; 
@btime aa = naive_flatten($a)
## built in function
@btime aa = collect(flatten($a))

  61.487 μs (2012 allocations: 222.64 KiB)
  204.570 μs (18015 allocations: 819.22 KiB)

```

Why is there such a big difference? My function shouldn’t even do well since I am not preallocating anything…

---

<div class="post-metadata">

**Author:** ![gpapo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gpapo/32/1857_2.png) [@gpapo](https://discourse.julialang.org/u/gpapo)\
**Post date:** [October 9, 2018, 9:07am UTC](https://discourse.julialang.org/t/custom-flatten-version-is-faster-than-flatten/16048/2 "2018-10-09T09:07:24Z")

</div>

I think that the problem is the current implementation of `Iterators.flatten` does not take advantage of many informations: first of all it treats arrays in a generic way, as we didn’t know their actual length, in fact if you see

[https://github.com/JuliaLang/julia/blob/0d713926f85dfa3e4e0962215b909b8e47e94f48/base/iterators.jl#L880-L897](https://github.com/JuliaLang/julia/blob/0d713926f85dfa3e4e0962215b909b8e47e94f48/base/iterators.jl#L880-L897)

`flatten_iteratorsize` does not specialize on `Union{HasLength, HasShape}` unless the iterator element type is a tuple or a number.

I don’t know if there is a deeper motivation about that.

---

<div class="post-metadata">

**Author:** ![mschauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mschauer/32/13946_2.png) [@mschauer](https://discourse.julialang.org/u/mschauer)\
**Post date:** [October 9, 2018, 10:53am UTC](https://discourse.julialang.org/t/custom-flatten-version-is-faster-than-flatten/16048/3 "2018-10-09T10:53:19Z")

</div>

This is not so much about `flatten_iteratorsize` but about the performance difference between

```julia
julia> function naive_flatten2(a)
           ret = Array{Float64}(undef, 0)
           for ai in a
               for aij in ai
                   push!(ret, aij)
               end
           end
           ret
       end
naive_flatten2 (generic function with 1 method)

julia> function naive_flatten3(a)
           ret = Array{Float64}(undef, 0)
           for ai in Iterators.flatten(a)
               push!(ret, ai)
           end 
           ret
       end
naive_flatten3 (generic function with 1 method)

julia> @btime aa = naive_flatten2($a);
  100.138 μs (14 allocations: 256.70 KiB)

julia> @btime aa = naive_flatten3($a);
  239.802 μs (18015 allocations: 819.22 KiB)

```

---

<div class="post-metadata">

**Author:** ![affans](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/affans/32/11911_2.png) [@affans](https://discourse.julialang.org/u/affans)\
**Post date:** [January 2, 2019, 3:55am UTC](https://discourse.julialang.org/t/custom-flatten-version-is-faster-than-flatten/16048/4 "2019-01-02T03:55:54Z")

</div>

Is this something worth opening an issue for?

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [January 2, 2019, 9:38am UTC](https://discourse.julialang.org/t/custom-flatten-version-is-faster-than-flatten/16048/5 "2019-01-02T09:38:39Z")

</div>

FWIW, I’d think it is.