# Pushing a large array into CircularDeque fails

**URL:** https://discourse.julialang.org/t/pushing-a-large-array-into-circulardeque-fails/10497
**Category:** New to Julia
**Created:** [April 23, 2018, 11:53pm UTC](https://discourse.julialang.org/t/pushing-a-large-array-into-circulardeque-fails/10497 "2018-04-23T23:53:08Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![Adriel](https://avatars.discourse-cdn.com/v4/letter/a/f07891/32.png) [@Adriel](https://discourse.julialang.org/u/Adriel)
#### Post date: [April 23, 2018, 11:53pm UTC](https://discourse.julialang.org/t/pushing-a-large-array-into-circulardeque-fails/10497/1 "2018-04-23T23:53:08Z")

</div>

Can anyone explain why the following fails?

```julia
julia> using DataStructures

julia> d = CircularDeque{Float64}(100_000)
CircularDeque{Float64}([])

julia> push!(d, randn(), randn()) # no problem
CircularDeque{Float64}([0.2451865901032298,0.3001460029950263])

julia> push!(d, randn(25_000)...) # wait 2 minutes...
ERROR: StackOverflowError:

```

I couldn’t find the root definition of `push!` for multiple values in Base, but I expect it’s a recursion along the _n_ input values, passing _n_-1 values to a new `push!` call at every stage. That would explain the excessive memory use.

I can easily move forward via a simple loop so it’s not a problem I need solved - I’m just wondering why `push!` is implemented this way and if it could be improved, e.g. with an iterative solution.

---

<div class="post-metadata">

### Author: ![mohamed82008](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mohamed82008/32/18171_2.png) [@mohamed82008](https://discourse.julialang.org/u/mohamed82008)
#### Post date: [April 24, 2018, 12:33am UTC](https://discourse.julialang.org/t/pushing-a-large-array-into-circulardeque-fails/10497/2 "2018-04-24T00:33:05Z")

</div>

> [@Adriel](#):
>
> `randn(25_000)...`

Splatting is bad for the function’s health, too many arguments to gulp down. ~~Try `append!` instead.~~

Edit: `append!` does not work. A good old loop does the trick though.

---

<div class="post-metadata">

### Author: ![TsurHerman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tsurherman/32/1234_2.png) [@TsurHerman](https://discourse.julialang.org/u/TsurHerman)
#### Post date: [April 24, 2018, 8:02am UTC](https://discourse.julialang.org/t/pushing-a-large-array-into-circulardeque-fails/10497/3 "2018-04-24T08:02:34Z")

</div>

Or use Broadcasting tricks:

`push!.([d], randn(25_000))`

---

<div class="post-metadata">

### Author: ![jlapeyre](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jlapeyre/32/4514_2.png) [@jlapeyre](https://discourse.julialang.org/u/jlapeyre)
#### Post date: [April 24, 2018, 11:25am UTC](https://discourse.julialang.org/t/pushing-a-large-array-into-circulardeque-fails/10497/4 "2018-04-24T11:25:34Z")

</div>

No, it has nothing to do with splatting:

```julia
function mypush!(a,items...)
    for i in items
        push!(a,i)
    end
    a
end

```

```julia
julia> using DataStructures;

julia> d = CircularDeque{Float64}(100_000);

julia> @time mypush!(d,randn(25_000)...);
  0.089943 seconds (113.93 k allocations: 6.478 MiB, 5.70% gc time)

julia> @time mypush!(d,randn(25_000)...);
  0.000831 seconds (25.01 k allocations: 781.625 KiB)

```

Another way to handle this is `reduce(push!,d,randn(25_000))`.

Or write special methods

```julia
function Base.push!(a::DataStructures.CircularDeque{T}, item::T) where T
    invoke(push!, Tuple{DataStructures.CircularDeque{T},Any}, a,item)
end

function Base.push!(a::DataStructures.CircularDeque{T}, item::T, items::T...) where T
    push!(a,item)
    for i in items
        push!(a,i)
    end
    a
end

```

This gives:

```julia
julia> @time push!(d,randn(25_000)...);
  0.002142 seconds (25.01 k allocations: 781.625 KiB)

```

```julia

```

---

<div class="post-metadata">

### Author: ![mohamed82008](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mohamed82008/32/18171_2.png) [@mohamed82008](https://discourse.julialang.org/u/mohamed82008)
#### Post date: [April 24, 2018, 11:35am UTC](https://discourse.julialang.org/t/pushing-a-large-array-into-circulardeque-fails/10497/5 "2018-04-24T11:35:43Z")

</div>

I stand corrected, it has to do with how the splatted arguments are treated inside, my bad.

---

<div class="post-metadata">

### Author: ![jlapeyre](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jlapeyre/32/4514_2.png) [@jlapeyre](https://discourse.julialang.org/u/jlapeyre)
#### Post date: [April 24, 2018, 11:49am UTC](https://discourse.julialang.org/t/pushing-a-large-array-into-circulardeque-fails/10497/6 "2018-04-24T11:49:39Z")

</div>

There are several functions like this. I used to think it was due to splatting. Even many of the most experienced people say the inefficiency is due to splatting too many arguments until they look into it more deeply.

Note that I am not saying that splatting 10^4 or 10^5 arguments is good or bad practice, just that it is not the cause of the huge inefficiency observed.

---

<div class="post-metadata">

### Author: ![tkoolen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkoolen/32/1603_2.png) [@tkoolen](https://discourse.julialang.org/u/tkoolen)
#### Post date: [April 24, 2018, 4:56pm UTC](https://discourse.julialang.org/t/pushing-a-large-array-into-circulardeque-fails/10497/7 "2018-04-24T16:56:08Z")

</div>

Vararg functions often (but not always) recurse. Recursing 25000 times leads to a stack overflow.
