So I am simply trying to find out how I identify that an iterator has come to the end of its content. The language is documentation standard doesn’t seem to be designed to help people understand what the contract of an interface is.
I come from a lot of use of Java amongst a bunch of other languages and I cannot stress enough how user unfriendly the documentation is.
I expect the documentation to explain all the landmark elements of an interface. So for example, in Java, if I look up the interface for an iterator it very clearly expand. Here’s the start. Here’s the end. Here’s how you query, whether or not the iterator is reach the end of its content, and these are all expressed as isclear statements and it takes a minute or two to understand what the interface does this is not the case with JL documentation.
I’ve spent like 20 minutes or a half an hour just trying to answer a question I know in every other language I’ve run into. I can get answered in about two minutes.
If I am using an interator on an array, how do I test that it’s reached the end?
Like if iterator.end() == true …?
But if somebody can explain to me what I’m supposed to do with the documentation system of the language in order to answer these questions you would be doing me a great favor
It’s, as raman_kumar notes, in the docs, under Interfaces->iterators, but note that iterators in julia typically are stateless. That is, the iterator itself does not “know” that it’s finished. There is usually no state which is updated in an iterator, so you can’t ask the iterator whether it has reached the end. The state is part of the iteration, not of the iterator. The most common way to use an iterator is in a for loop. You don’t get to see the iteration state in a for loop, only the elements, one by one. And there is no general way to tell if you’re in the last iteration. It depends on the iterator. Some iterators have a known length, whereas others don’t, and some go on indefinitely.
I want to point out, the structure of my complaint is that the documentation is there, but it is so organized as to be nearly useless or extremely frustrating to somebody like myself..
For example, if I go to C++ or to Java or any other language documentation, and they describe an interface like iterators, you see a clear, concise definition of what they are on a single page and what the contract for that interface is.
My experience with JL documentation is it not clearly presented and it’s so obscurely presented as to require an order of magnitude more effort to learn about many things that are basic.
This thread is a testimony to how frustrating it is. The documentation assumes people don’t care about the constituent parts and it assumes people will only use them in a loop or a while loop and the mechanics will be handled by the language.
I see your point. An inconvenience with julia, reflected in the documentation, is that very little is “part of the language”. The interfaces are more or less just conventions. Iterators are sort of an exception, in that the compiler/parser/lowering-engine actually rewrites for loops (literally) into while loops, so this particular interface is in a sense part of the language. But the other “interfaces” are merely conventions. Even conversion of 1 + 1.0 to Float64 is sort of a convention. It can be changed at will.
Every function and operator can be overloaded, and it’s common to do so, it’s even sort of how the entire language operates. (Try e.g. a methods(*) to see it). And it’s a kind of a silent agreement that you shouldn’t abuse things (e.g. so that a + b suddenly means "print a, save b to a file, and return b - a).
Of course, such things are also possible in other languages, but overloading/dispatch is typically not done as massively at the user level as in julia. I suspect it’s simply too much functionality to document properly.
In a sense, every abstract type, there are many of them, is a sort of interface, but little is documented about what an abstract type promises in terms of functionality:
help?> IO
search: IO
No documentation found for public binding Core.IO.
...
help?> Number
search: Number Timer outer
Number
Abstract supertype for all number types.
Partly because it’s not necessarily well defined. It’s been introduced to collect some subtypes, without too much thought about what it really means in terms of functionality.
Does it not? To me this from the manual is quite clear:
Required method Brief description
iterate(iter) Returns either a tuple of the first item and initial state or nothing if empty
iterate(iter, state) Returns either a tuple of the next item and next state or nothing if no items remain
So if the iterator is exhausted then calling iterate(iter, state) returns nothing.
Could you perhaps elaborate why you find this unclear and perhaps we can try to improve/clear up the wording?
unfortunately, it is not just an issue of documentation. interfaces in Julia are generally underspecified. Contracts that these interfaces satisfy are generally defined more by convention, trial-and-error, and word of mouth, than they are defined by an explicit and exhaustive design doc like you would find in C++ or Java.
in the case of iteration specifically, I think really all that can be said is
iterate(x) must return a value and the next state (or nothing)
iterate(x, state) must return a value and the next state (or nothing)
when nothing is returned the iterator is done
but otherwise there are basically no universal rules about things like
is iterate(x, state) idempotent
answer: no e.g. for stateful iterators
does the number of iterate calls before nothing have to match length(x) ? and similarly does eltype(x) have to actually match the types of the values you get from iterate(x)
answer: it really should if you don’t want to break things (collect, map) in horrible ways, but the interface doesn’t explicitly demand this.
if x == y, is some state_x obtained from iterating x a valid iteration state for y ?
in practice, usually yes, but the interface doesn’t demand it so it theoretically might not be. but using an invalid state for an iterator can be UB
what the shape of a value and the next state should be. I was intentionally ambiguous with that wording since there is no actual requirement that it’s a Tuple. it just has to be destructurable into two parts like x, state = iterate(x)
that being said, I still think in practice this all ends up working out pretty well. I have found the iteration primitives & utilities to be powerful & basically sufficient for implementing any kind of iterator needed. so I am not trying to be negative — mostly just highlight that your frustration is very reasonable, but it is not caused so much by people “not caring about constituent parts” as it is by those parts being pretty amorphous in the first place.
Not really. A for loop results in exactly the same lowered code as the equivalent while loop. I.e. the intermediate representations are identical except for some vaiable names.
It’s just how a for loop is defined, as fully equivalent to the while loop in the docs about iterators (which you linked to above). Even though no textual while is inserted during the translation, it’s lowered to the same intermediate represenation. So, in this sense, a for loop is just syntactic sugar for a slightly more verbose while loop.
I suppose that’s an impementation detail. Whether it’s translated into a textual while loop, or translated into a functionally equivalent intermediate representation can’t really confuse anyone?