`isempty(enumerate(Iterators.Stateful(...)))` advances the stateful iterator?

On Julia 1.6.2:

itr = enumerate(Iterators.Stateful([:a, :b, :c]))
isempty(itr) && println("It's empty")
isempty(itr) && println("It's empty")
isempty(itr) && println("It's empty")
isempty(itr) && println("It's empty")
It's empty

By contrast, if we remove the enumerate, the above prints nothing and evaluates to false. Is this intended behavior or is it a bug?

Well, stateful iterators are a bit of an underspecified part of the iteration protocol (along with a few other issues), but rather than asking whether this is a bug, I think the better question is whether this can be fixed (yes) and whether this should be fixed (sure, why not). The relevant code is here:
https://github.com/JuliaLang/julia/blob/d89addd21b97865eb5445f0051e5946d13842c68/base/essentials.jl#L788

In particular, isdone needs to be implemented for the enumerate iterator and just pass through to the iterator it wraps. Should be a quick PR for someone who wants to send one in :).

2 Likes

Awesome, thanks @Keno for looking into this!

rather than asking whether this is a bug, I think the better question is whether this can be fixed (yes) and whether this should be fixed (sure, why not).

FWIW, I would not be able to ask (let alone answer) whether something can or should be fixed without first knowing whether it’s a bug, which is nontrivial because it requires knowing what the intended behavior is. Hence my asking whether that was the intended behavior – it seemed quite possible that someone would come back with “It sure is, here’s why.”

Much appreciate the link to the relevant code. And I see in the docstring of isdone they do say:

This function provides a fast-path hint for iterator completion. This is
useful for mutable iterators that want to avoid having elements consumed, if
they are not going to be exposed to the user (e.g. to check for done-ness in
isempty or zip). Mutable iterators that want to opt into this feature
should define an isdone method that returns true/false depending on whether
the iterator is done or not. Stateless iterators need not implement this
function. If the result is missing, callers may go ahead and compute
iterate(x, state...) === nothing to compute a definite answer.

i.e., just the info I needed as the implementer of a custom iterator :slight_smile: So I guess the bug is that information, which is important for someone who wants to implement a mutable custom iterator, is in the docstring for a non-exported function and doesn’t appear on the Julia documentation site. It should probably be at least mentioned on the Iteration interface page. That I feel I know enough to submit a fix for :slight_smile:

1 Like

Filed https://github.com/JuliaLang/julia/pull/43099.

1 Like