How to apply `Iterators.drop` on an `Iterators.Stateful`?

General Usage
1

See the following MWE:

julia> a = Iterators.Stateful("123456789");

julia> b = Iterators.drop(a, 3)
Base.Iterators.Drop{Base.Iterators.Stateful{String,Union{Nothing, Tuple{Char,Int64}}}}(Base.Iterators.Stateful{String,Union{Nothing, Tuple{Char,Int64}}}("123456789", ('1', 2), 0), 3)

julia> iterate(b)
('4', nothing)

julia> iterate(b)
('8', nothing)

julia> iterate(b)
# nothing

Variable b drops the first 3 elements of a every time iterating it.
But what I want is actually drop the “123” and iterate a one by one.
A working way could be

julia> a = Iterators.Stateful("123456789");

julia> b = Iterators.Stateful(collect(Iterators.drop(a, 3)))
Base.Iterators.Stateful{Array{Char,1},Union{Nothing, Tuple{Char,Int64}}}(['4', '5', '6', '7', '8', '9'], ('4', 2), 0)

julia> iterate(b)
('4', nothing)

julia> iterate(b)
('5', nothing)

julia> iterate(b)
('6', nothing)
...

But if a is large of infinitive, collecting is not possible.

Of course, I could write, in this example,

julia> a = Iterators.Stateful(Iterators.drop("123456789", 3));

julia> iterate(a)
('4', nothing)

julia> iterate(a)
('5', nothing)

julia> iterate(a)
('6', nothing)

from the beginning, but sometimes this is not possible. I am given a as a Stateful already.

2

Maybe Iterators.rest? See here.

What’s crucial is understanding that each function from Iterators returns a new iterator wrapping whatever you gave to it and that nested iterators are applied one after the other.

3

It seems to work:

julia> a = Iterators.Stateful("123456789")
Base.Iterators.Stateful{String,Union{Nothing, Tuple{Char,Int64}}}("123456789", ('1', 2), 0)

julia> b = Iterators.drop(a, 3)
Base.Iterators.Drop{Base.Iterators.Stateful{String,Union{Nothing, Tuple{Char,Int64}}}}(Base.Iterators.Stateful{String,Union{Nothing, Tuple{Char,Int64}}}("123456789", ('1', 2), 0), 3)

julia> for x in Iterators.rest(b)
           println(x)
       end
4
5
6
7
8
9

Thank you!

4

I was thinking more along the lines of Iterators.rest(a,3) instead of drop altogether, since that is the explicit purpose of the rest iterator.

5

Oh, I guess it does not work so simple for a Stateful:

julia> a = Iterators.Stateful("123456789")
Base.Iterators.Stateful{String,Union{Nothing, Tuple{Char,Int64}}}("123456789", ('1', 2), 0)

julia> for x in Iterators.rest(a, 3)
           println(x)
       end
1
2
3
4
5
6
7
8
9

It does work if a is just a normal iterable:

julia> a = "123456789"
"123456789"

julia> for x in Iterators.rest(a, 3)
           println(x)
       end
3
4
5
6
7
8
9
6

Stumbling over this and seeing it as unsolved given that the marked solution doesn’t help to understand the original problem, I feel the urge to correct a few things:

  1. Iterators.rest is not for skipping or dropping elements. I’d even advise against using Iterators.rest unless you know exactly how the iterator tracks state because the 2nd argument of the rest function is a state object. Having that work here is coincidental luck, since for strings and arrays, state is tracked by index. But in case of the Stateful iterator, the 2nd argument doesn’t have any effect since Stateful iterators don’t need it to work given the internal state (though, they need a 2nd argument in order to fulfill the protocol).

  2. The only problem in the initial MWE was that you call the single argument iterate function whose sole purpose is to start an iteration (which in the case of drop drops n values).
    So for all follow-up calls to iterate there should be a 2nd argument. Or worded differently, proper use of the iteration protocol works flawlessly here.

iter = Iterators.drop(Iterators.Stateful("123456789"), 3)
for x in iter
  println(x)
end

should work as intended.
if you want to step over it manually, then you need to feed the state of the previous iteration into the iterate function:

iter = Iterators.drop(Iterators.Stateful("123456789"), 3)
next = iterate(iter)
while next!==nothing
  elem, state = next
  println(elem)
  next = iterate(iter, state)
end

Sorry for reviving it. But given that one can still stumble over this (as I did) I felt the urge to correct some aspects :see_no_evil:

2 Likes