# Iterators.Stateful is not an iterator?

**URL:** <https://discourse.julialang.org/t/iterators-stateful-is-not-an-iterator/53080>\
**Category:** New to Julia\
**Created:** [January 9, 2021, 11:28am UTC](https://discourse.julialang.org/t/iterators-stateful-is-not-an-iterator/53080 "2021-01-09T11:28:26Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![freeman](https://avatars.discourse-cdn.com/v4/letter/f/ec9cab/32.png) [@freeman](https://discourse.julialang.org/u/freeman)\
**Post date:** [January 9, 2021, 11:28am UTC](https://discourse.julialang.org/t/iterators-stateful-is-not-an-iterator/53080/1 "2021-01-09T11:28:26Z")

</div>

I’m trying to understand iterators better because I’m trying to decide whether I should, for my specific application, define `iterate` for my type or define a constructor of `Iterators.Stateful` for my type. And I seem to have reached the conclusion that `Iterators.Stateful` aren’t iterators?

The interfaces page [[1]](https://docs.julialang.org/en/v1/manual/interfaces/) says that to implement the interator interface, two methods are required:

- `iterate(iter)` Returns either a tuple of the first item and initial state or [`nothing`](https://docs.julialang.org/en/v1/base/constants/#Core.nothing) if empty
- `iterate(iter, state)` Returns either a tuple of the next item and next state or `nothing` if no items remain

So a vector is an iterator:

```julia
x = ["a", "b", "c"];
iterate(x) # returns ("a", 2) - state 2 indexes the next element
iterate(x, 2) # returns ("b", 3)
iterate(x, 3) # returns ("c", 4)
iterate(x, 4) # returns nothing

```

What happens if we make a stateful iterator out of x?

```julia
it = Iterators.Stateful(x);
iterate(it) # returns ("a", nothing) - no state?
iterate(it) # returns ("b", nothing) - no state?
iterate(it) # returns ("c", nothing) - no state?
iterate(it) # returns nothing - this one is correct

```

As a separate point, I’ve found this git issue which seems to show an enormous bug: [https://github.com/JuliaLang/julia/issues/35530](https://github.com/JuliaLang/julia/issues/35530)

So my question is: Am I misunderstanding something? And if not, should I conclude that I shouldn’t use `Iterators.Stateful` as it looks kind of broken / abandoned?

---

<div class="post-metadata">

**Author:** ![simeonschaub](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/simeonschaub/32/216566_2.png) [@simeonschaub](https://discourse.julialang.org/u/simeonschaub)\
**Post date:** [January 9, 2021, 11:50am UTC](https://discourse.julialang.org/t/iterators-stateful-is-not-an-iterator/53080/2 "2021-01-09T11:50:31Z")

</div>

No, this is definitely expected behavior. `Stateful` stores the iteration state inside the `Stateful` object, so there is no reason to pass any additional state around here.

---

<div class="post-metadata">

**Author:** ![freeman](https://avatars.discourse-cdn.com/v4/letter/f/ec9cab/32.png) [@freeman](https://discourse.julialang.org/u/freeman)\
**Post date:** [January 9, 2021, 11:51am UTC](https://discourse.julialang.org/t/iterators-stateful-is-not-an-iterator/53080/3 "2021-01-09T11:51:55Z")

</div>

I understand there’s no reason to, but it still doesn’t agree with interface spec…

---

<div class="post-metadata">

**Author:** ![simeonschaub](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/simeonschaub/32/216566_2.png) [@simeonschaub](https://discourse.julialang.org/u/simeonschaub)\
**Post date:** [January 9, 2021, 11:52am UTC](https://discourse.julialang.org/t/iterators-stateful-is-not-an-iterator/53080/4 "2021-01-09T11:52:39Z")

</div>

> [@freeman](#):
>
> As a separate point, I’ve found this git issue which seems to show an enormous bug: [Stateful Iterator omitting last element when used in list comprehension, crashes on 1-element long vectors. · Issue #35530 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/issues/35530)

Yes, this is a bug, as pointed out in the issue, since `collect` fails to take into account that an iterator with defined length could also be stateful

---

<div class="post-metadata">

**Author:** ![simeonschaub](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/simeonschaub/32/216566_2.png) [@simeonschaub](https://discourse.julialang.org/u/simeonschaub)\
**Post date:** [January 9, 2021, 11:55am UTC](https://discourse.julialang.org/t/iterators-stateful-is-not-an-iterator/53080/5 "2021-01-09T11:55:05Z")

</div>

It perfectly agrees with that for me:

```julia
julia> it = Iterators.Stateful([1, 2, 3]);

julia> iterate(it)
(1, nothing)

julia> iterate(it, ans[2])
(2, nothing)

julia> iterate(it, ans[2])
(3, nothing)

julia> iterate(it, ans[2]) === nothing
true

```

---

<div class="post-metadata">

**Author:** ![freeman](https://avatars.discourse-cdn.com/v4/letter/f/ec9cab/32.png) [@freeman](https://discourse.julialang.org/u/freeman)\
**Post date:** [January 9, 2021, 12:00pm UTC](https://discourse.julialang.org/t/iterators-stateful-is-not-an-iterator/53080/6 "2021-01-09T12:00:07Z")

</div>

Ok fine different readings. I read “Returns either a tuple of the next item and next state” as meaning that the “next state” actually controls what is returned by applying `iterate(x, state)`, you read it as meaning that’s not necessarily the case.

---

<div class="post-metadata">

**Author:** ![simeonschaub](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/simeonschaub/32/216566_2.png) [@simeonschaub](https://discourse.julialang.org/u/simeonschaub)\
**Post date:** [January 9, 2021, 12:07pm UTC](https://discourse.julialang.org/t/iterators-stateful-is-not-an-iterator/53080/7 "2021-01-09T12:07:10Z")

</div>

Yes, `iterate` doesn’t necessarily have to be pure, so you always need `state` as well as the object (and perhaps even additional global state, as in `eachline()`) to predict what the next item will be and the object may also be mutated during `iterate`. Perhaps the docs could be a bit clearer here, feel free to make a PR, if you think this could be explained better.

---

<div class="post-metadata">

**Author:** ![freeman](https://avatars.discourse-cdn.com/v4/letter/f/ec9cab/32.png) [@freeman](https://discourse.julialang.org/u/freeman)\
**Post date:** [January 9, 2021, 12:13pm UTC](https://discourse.julialang.org/t/iterators-stateful-is-not-an-iterator/53080/8 "2021-01-09T12:13:35Z")

</div>

> [@simeonschaub](#):
>
> and the object may also be mutated during `iterate`

Exactly, that’s another point. When I first read “Returns either a tuple of the next item and next state” I took it as implying “… and therefore the object doesn’t have to be mutated”. Now it doesn’t surprise me that this aspect is false for the case of stateful iterators - their name indicates as much. But now that you spelled it out, I realized: since `iterate` doesn’t contain an exclamation mark `!` in its name, I implicitly assumed it wouldn’t mutate (again, expect for stateful iterators).

Anyway, thanks for the clarification.

---

<div class="post-metadata">

**Author:** ![yashi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yashi/32/19448_2.png) [@yashi](https://discourse.julialang.org/u/yashi)\
**Post date:** [March 1, 2022, 2:24pm UTC](https://discourse.julialang.org/t/iterators-stateful-is-not-an-iterator/53080/9 "2022-03-01T14:24:52Z")

</div>

Thank you @freeman and @simeonschaub for the explanations. I was also confused by the fact that `iterate(x, state)` doesn’t have `!` as @freeman said. And I admit that I wasn’t careful enough to read the following sentences in [the page](https://docs.julialang.org/en/v1/manual/interfaces/) (emphasis mine).

> Instead of mutating objects as they are iterated over, Julia iterators _ **may** _ keep track of the iteration state externally from the object.

To enhance the doc, how about adding:

> `state` may or may not be used depending on `iter`.

to the “Brief description” of `iterate(x, state)`?

And add the following example (taken from @simeonschaub 's)

```julia
julia> it = Iterators.Stateful([1, 2, 3]);

julia> iterate(it, nothing)
(1, nothing)

julia> iterate(it, nothing)
(2, nothing)

julia> iterate(it, nothing)
(3, nothing)

```

If the general direction is OK, I’ll prepare a PR for the doc.