# \`cat\` allocates too much memory

**URL:** <https://discourse.julialang.org/t/cat-allocates-too-much-memory/50443>\
**Category:** Performance\
**Tags:** question\
**Created:** [November 19, 2020, 3:03pm UTC](https://discourse.julialang.org/t/cat-allocates-too-much-memory/50443 "2020-11-19T15:03:53Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![ArthurW](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/arthurw/32/19561_2.png) [@ArthurW](https://discourse.julialang.org/u/ArthurW)\
**Post date:** [November 19, 2020, 3:03pm UTC](https://discourse.julialang.org/t/cat-allocates-too-much-memory/50443/1 "2020-11-19T15:03:53Z")

</div>

Hi. I’m new to Julia language.

I’m trying to convert a big vector of small arrays into a tensor, but `cat()` is drawing too much resources.

```julia
julia> A = [rand(Int, (28,28)) for _ ∈ 1:10000];

julia> @time cat(A..., dims=3);
  4.226158 seconds (50.11 M allocations: 3.044 GiB, 30.98% gc time)

```

Oddly, making the operation more explicit dramatically improves this:

```julia
julia> stacked = Array{Int, 3}(undef, 28, 28, 10000);

julia> @time map!.(x->x, eachslice(stacked, dims=3), A);
  0.133267 seconds (222.95 k allocations: 11.972 MiB)

```

[This reply to another topic](https://discourse.julialang.org/t/multiply-dense-arrays/47724/5) mentions that `reduce(hcat, A)` and `reduce(vcat, A)` are optimized to perform better than `hcat(A...)` and `vcat(A...)`. Is there such thing for `cat()`?

The following naive attempt made things worse:

```julia
julia> @time reduce((x...)->cat(x..., dims=3), A);
  4.431142 seconds (616.15 k allocations: 18.539 GiB, 1.53% gc time)

```

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [November 19, 2020, 3:25pm UTC](https://discourse.julialang.org/t/cat-allocates-too-much-memory/50443/2 "2020-11-19T15:25:49Z")

</div>

it’s not cat that is allocating, it’s the splatting

---

<div class="post-metadata">

**Author:** ![rdeits](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rdeits/32/286_2.png) [@rdeits](https://discourse.julialang.org/u/rdeits)\
**Post date:** [November 19, 2020, 3:28pm UTC](https://discourse.julialang.org/t/cat-allocates-too-much-memory/50443/3 "2020-11-19T15:28:36Z")

</div>

You’ll probably get the best possible performance by doing this in the simplest way: Construct a new array of the appropriate size and then write a loop that fills in each slice of the stacked array. One-liner solutions are convenient, but a less-clever approach often offers the best performance.

---

<div class="post-metadata">

**Author:** ![ArthurW](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/arthurw/32/19561_2.png) [@ArthurW](https://discourse.julialang.org/u/ArthurW)\
**Post date:** [November 19, 2020, 3:32pm UTC](https://discourse.julialang.org/t/cat-allocates-too-much-memory/50443/4 "2020-11-19T15:32:26Z")

</div>

Is there a way to bypass that?

This solution doesn’t use splatting and perform even worse:

```julia
julia> @time reduce((x,y)->cat(x, y, dims=3), A);
  6.262270 seconds (526.79 k allocations: 18.537 GiB, 3.85% gc time)

```

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [November 19, 2020, 3:38pm UTC](https://discourse.julialang.org/t/cat-allocates-too-much-memory/50443/5 "2020-11-19T15:38:20Z")

</div>

this allocate once for ever element so even more allocation. Do what @rdeits suggested I think.

---

<div class="post-metadata">

**Author:** ![rdeits](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rdeits/32/286_2.png) [@rdeits](https://discourse.julialang.org/u/rdeits)\
**Post date:** [November 19, 2020, 3:40pm UTC](https://discourse.julialang.org/t/cat-allocates-too-much-memory/50443/6 "2020-11-19T15:40:30Z")

</div>

Yeah, the loop is more than 100X faster than most of the options above and allocates vastly less memory:

```julia
julia> function assemble(A)
         stacked = Array{Int, 3}(undef, size(first(A))..., length(A))
         for i in 1:length(A)
           stacked[:, :, i] = A[i]
         end
         stacked
       end
assemble (generic function with 1 method)

julia> using BenchmarkTools

julia> A = [rand(Int, (28,28)) for _ ∈ 1:10000];

julia> @btime assemble($A);
  24.793 ms (2 allocations: 59.81 MiB)

```

---

<div class="post-metadata">

**Author:** ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)\
**Post date:** [November 19, 2020, 3:41pm UTC](https://discourse.julialang.org/t/cat-allocates-too-much-memory/50443/7 "2020-11-19T15:41:17Z")

</div>

here’s a version using an optimized reduction and a reshape

```julia
julia> reshape(reduce(hcat, A), size(A[1])..., :) == cat(A..., dims=3)
true

julia> @time reshape(reduce(hcat, A), size(A[1])..., :);
  0.028446 seconds (7 allocations: 59.815 MiB)

```

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [November 19, 2020, 3:49pm UTC](https://discourse.julialang.org/t/cat-allocates-too-much-memory/50443/8 "2020-11-19T15:49:06Z")

</div>

> [@ArthurW](#):
>
> ```julia
> julia> @time reduce((x,y)->cat(x, y, dims=3), A);
> 6.262270 seconds (526.79 k allocations: 18.537 GiB, 3.85% gc time)
> 
> ```

So why is `cat` so much slower than `vcat`:

```julia
julia> @time reduce((x,y)->cat(x, y;dims=3), A);
 12.606505 seconds (526.91 k allocations: 18.537 GiB, 16.69% gc time)

julia> @time reduce(vcat, A);
  0.038854 seconds (2 allocations: 59.815 MiB)

```

?

---

<div class="post-metadata">

**Author:** ![rdeits](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rdeits/32/286_2.png) [@rdeits](https://discourse.julialang.org/u/rdeits)\
**Post date:** [November 19, 2020, 3:56pm UTC](https://discourse.julialang.org/t/cat-allocates-too-much-memory/50443/9 "2020-11-19T15:56:03Z")

</div>

`reduce(vcat, ...)` and `reduce(hcat, ...)` are fast because they have specialized implementations: [https://github.com/JuliaLang/julia/blob/e402cf47dd8e3c509969c90c38ad5d57c746eccf/base/abstractarray.jl#L1564-L1568](https://github.com/JuliaLang/julia/blob/e402cf47dd8e3c509969c90c38ad5d57c746eccf/base/abstractarray.jl#L1564-L1568) . In other words, they produce the same result as naively calling `vcat` on each element, but they do so in a way which is much more efficient than the naive approach (essentially by performing the same allocation and then loop operation that I wrote out by hand).

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [November 19, 2020, 4:07pm UTC](https://discourse.julialang.org/t/cat-allocates-too-much-memory/50443/10 "2020-11-19T16:07:20Z")

</div>

It seems like a special implementation for `cat` would be natural. Is that perhaps more difficult to do, since `(x, y)->cat(x, y;dims=n)` is anonymous?

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [November 19, 2020, 4:08pm UTC](https://discourse.julialang.org/t/cat-allocates-too-much-memory/50443/11 "2020-11-19T16:08:32Z")

</div>

yeah I think the `dims` info is gone.

---

<div class="post-metadata">

**Author:** ![mcabbott](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mcabbott/32/6603_2.png) [@mcabbott](https://discourse.julialang.org/u/mcabbott)\
**Post date:** [November 19, 2020, 4:50pm UTC](https://discourse.julialang.org/t/cat-allocates-too-much-memory/50443/12 "2020-11-19T16:50:54Z")

</div>

> [@DNF](#):
>
> a special implementation for `cat` would be natural. Is that perhaps more difficult to do, since `(x, y)->cat(x, y;dims=n)` is anonymous?

This PR [Allow `reduce(cat(dims=4), A)`, with efficient method for simple cases by mcabbott · Pull Request #37196 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/pull/37196) wants to make `cat(dims=3)::Base.Fix1` to enable precisely this, and then make `reduce(cat(dims=3), A)` efficient. For now @baggepinnen’s answer is probably the way to go.

Unless you want to use a package, in which case you can make a view of them, instead of a new dense array. These should be fast to construct but possibly slower to use, in whatever the next step is.

```julia
julia> B = reduce(cat(dims=3), A);

julia> B ≈ JuliennedArrays.Align(A, 1,2)
true

julia> B ≈ LazyStack.stack(A)
true

julia> B ≈ RecursiveArrayTools.VectorOfArray(A)
true

```