# Creating an iterator by wrapping another one

**URL:** <https://discourse.julialang.org/t/creating-an-iterator-by-wrapping-another-one/22280>\
**Category:** Performance\
**Created:** [March 24, 2019, 5:47pm UTC](https://discourse.julialang.org/t/creating-an-iterator-by-wrapping-another-one/22280 "2019-03-24T17:47:15Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![fverdugo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fverdugo/32/9446_2.png) [@fverdugo](https://discourse.julialang.org/u/fverdugo)\
**Post date:** [March 24, 2019, 5:47pm UTC](https://discourse.julialang.org/t/creating-an-iterator-by-wrapping-another-one/22280/1 "2019-03-24T17:47:15Z")

</div>

I am trying to create an iterator that wraps another one. Let us assume that we have this iterator that returns `l` times the array `a` :

```julia
struct Foo
    a::Array{Float64,1}
    l::Int
end

Base.iterate(foo::Foo) = iterate(foo,0)
function Base.iterate(foo::Foo,state::Int)
    if foo.l == state
        nothing
    else
        k = state + 1
        (foo.a,k)
    end
end

```

My goal is to implement another iterator that wraps previous one:

```julia
struct Bar
    foo::Foo
end

function Base.iterate(bar::Bar)
    foonext = iterate(bar.foo)
    iterate(bar,foonext)
end
function Base.iterate(bar::Bar,state)
    if state == nothing
        nothing
    else
        (fooi,foostate) = state
        foonext = iterate(bar.foo,foostate)
        (fooi,foonext)
    end
end

```

Iterating over instances of `Foo` works fine (as expected):

```julia
function run(x)
    for xi in x
    end
end
foo = Foo([1.2,3.2,1.1],10000000)
@time run(foo)
@time run(foo)

```

```julia
  0.288515 seconds (12.11 k allocations: 750.146 KiB)
  0.274807 seconds (4 allocations: 160 bytes)

```

The problems is that by iterating instances of `Bar` I get unwanted memory allocation:

```julia
bar = Bar(foo)
@time run(bar)
@time run(bar)

```

```julia
  0.795432 seconds (10.02 M allocations: 306.389 MiB, 3.56% gc time)
  0.669443 seconds (10.00 M allocations: 305.176 MiB, 0.99% gc time)

```

I would say that the code above is type stable…what I am missing?

Thanks in advance for the help!

---

<div class="post-metadata">

**Author:** ![jbrea](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jbrea/32/3879_2.png) [@jbrea](https://discourse.julialang.org/u/jbrea)\
**Post date:** [March 25, 2019, 9:05am UTC](https://discourse.julialang.org/t/creating-an-iterator-by-wrapping-another-one/22280/2 "2019-03-25T09:05:34Z")

</div>

I think it is not type stable because `foonext` can be `nothing`. You may want to check with `@code_warntype iterate(bar)`.

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [March 25, 2019, 9:52am UTC](https://discourse.julialang.org/t/creating-an-iterator-by-wrapping-another-one/22280/3 "2019-03-25T09:52:43Z")

</div>

Could do something like

```julia
function Base.iterate(bar::Bar)
    foonext = iterate(bar.foo)
    foonext === nothing && return nothing
    return iterate(bar, foonext)
end

function Base.iterate(bar::Bar, state)
    fooi, foostate = state
    foonext = iterate(bar.foo, foostate)
    foonext === nothing && return nothing
    return fooi, foonext
end

julia> @time run(bar)
  0.000003 seconds (4 allocations: 160 bytes)

```

Basically, every time we call `iterate` on the inner iterator we check if it is `nothing` (with `===`) and handle it. The compiler can then assume from thereon that the return value of the iterator is not `nothing`.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [March 25, 2019, 10:00am UTC](https://discourse.julialang.org/t/creating-an-iterator-by-wrapping-another-one/22280/4 "2019-03-25T10:00:46Z")

</div>

> [@kristoffer.carlsson](#):
>
> every time we call `iterate` on the inner iterator we check if it is `nothing` (with `===` ) and handle it

[`IterTools.@ifsomething`](https://juliacollections.github.io/IterTools.jl/latest/#IterTools.@ifsomething-1) is a macro specifically for this pattern.

---

<div class="post-metadata">

**Author:** ![ndinsmore](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ndinsmore/32/7433_2.png) [@ndinsmore](https://discourse.julialang.org/u/ndinsmore)\
**Post date:** [March 25, 2019, 12:19pm UTC](https://discourse.julialang.org/t/creating-an-iterator-by-wrapping-another-one/22280/5 "2019-03-25T12:19:28Z")

</div>

A Further optimization is to check for nothing before and after the inner iterate and `return fooi, foostate` or `nothing`.

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [March 25, 2019, 12:21pm UTC](https://discourse.julialang.org/t/creating-an-iterator-by-wrapping-another-one/22280/6 "2019-03-25T12:21:58Z")

</div>

What do you mean before the inner `innerate`? Can you write out the code?

---

<div class="post-metadata">

**Author:** ![fverdugo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fverdugo/32/9446_2.png) [@fverdugo](https://discourse.julialang.org/u/fverdugo)\
**Post date:** [March 25, 2019, 1:46pm UTC](https://discourse.julialang.org/t/creating-an-iterator-by-wrapping-another-one/22280/7 "2019-03-25T13:46:57Z")

</div>

Thanks for the answer!

It almost fixes it. However, the suggested implementation of `Bar` leads to instances that iterate one iteration less that the wrapped instance of `Foo`.

---

<div class="post-metadata">

**Author:** ![fverdugo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fverdugo/32/9446_2.png) [@fverdugo](https://discourse.julialang.org/u/fverdugo)\
**Post date:** [March 25, 2019, 1:58pm UTC](https://discourse.julialang.org/t/creating-an-iterator-by-wrapping-another-one/22280/8 "2019-03-25T13:58:29Z")

</div>

I have it!

It can be done very simply by using the `foostate` directly instead of creating a new state.

```julia
function Base.iterate(bar::Bar)
    foonext = iterate(bar.foo)
    foonext === nothing && return nothing
    foonext
end
function Base.iterate(bar::Bar,foostate)
    foonext = iterate(bar.foo,foostate)
    foonext === nothing && return nothing
    foonext
end

julia> @time run(bar)
  0.000003 seconds (4 allocations: 160 bytes)

```

Thanks for all your comments!

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [March 25, 2019, 2:42pm UTC](https://discourse.julialang.org/t/creating-an-iterator-by-wrapping-another-one/22280/9 "2019-03-25T14:42:15Z")

</div>

> [@fverdugo](#):
>
> ```julia
> function Base.iterate(bar::Bar)
> foonext = iterate(bar.foo)
> foonext === nothing && return nothing
> foonext
> end
> 
> ```

This is equivalent to

```julia
Base.iterate(bar::Bar) = iterate(bar.foo)

```

---

<div class="post-metadata">

**Author:** ![fverdugo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/fverdugo/32/9446_2.png) [@fverdugo](https://discourse.julialang.org/u/fverdugo)\
**Post date:** [March 26, 2019, 7:00am UTC](https://discourse.julialang.org/t/creating-an-iterator-by-wrapping-another-one/22280/10 "2019-03-26T07:00:43Z")

</div>

Yes, thanks for noting!

At the end, the solution to my first question is very simple: One has to wrap the calls to `iterate` . That’s all.