How do you actually find out *how* to use an interface in Julia, like the concept of iterator "end"?

New to Julia
21

search “java iterator”

boom. done. You have a direct path to all the iterators and their construction and usage.

c++ iterator

search “julia iterator”

using the JL internal search yields a mess (discussed shortly)

I found the

next = iterate(iter)
while next !== nothing
    (i, state) = next
    # body
    next = iterate(iter, state)
end

example in substack after 20 minutes of poking around in JL documentation. Even so, I had to fiddle with search terms in order to find the question and answer with the usage example you presented.

discussion:

My chief difficulty is I need to know in advance where the topic is filed under.

If you go to the base documentation

and type “iterators” then click on the the hopefully named Base.iterators all the reader is presented with is “there’s a module named iterators”

fine.

The side panel is dominated ESSENTIALS section presented contains a long list of not the thing I need. Below it, obscured under the mass of essentials, is Collections and Data Structures, followed Mathematics, Numbers which are all not the topic I am searching for. In terms of search results that side panel quickly looks like it’s not relative relevant to the search I just made.

The distinguishing factor in the first two language examples is I am presented with information that allows me to do the thing I need to.

In JL the usage is explained in passing

Further, if I had known to search for iterate(), I would still have to infer the initialization as

iterate(thing)

and know the unpacking convention

(i, state) = next

AND iterate(thing, state) means next item

The actual documentation for iterate() leaves almost everything unsaid.

If my discussion here sounds a bit frustrated and cranky. It’s because it shouldn’t be this hard and hopefully is a useful explanation of why I found everything and find everything relatively more challenging to learn about in Julia. The smart alec way I would describe it is that JL documentation is orthogonal to almost all my expectations. With every other language I have worked with the chief advantage in beauty of Java was that the language designers decided to include a very direct template for class documentation for the fields for the methods for the arguments of the methods and an entire system that was used to generate a useful documentation tree so people could understand abstractions down to their implementations all provided there. I should know I worked with some of the principal people who developed the language. They put a lot of thought into this into making the programming environment of Java, deliberately accessible to what they would call the average programmer. They made it pretty darn clear and pretty darn easy also aided by the structure of the language itself to be quite explanatory in a very concise way.

1 Like
22

Probably because you should start with reading the manual first. (Just “Manual”, not the standard library, the devdocs, etc). Searching on demand for every single bit of information is going to get frustrating because you are not familiar with key concepts, so you lack a foundation.

It is like learning linear algebra by trying to search for “SVD” first.

1 Like
23

I personally find the Java documentation particularly obtuse for newcomers as I illustrated above, but I know my way around it from experience. In particular, I know Java and C++ are object oriented so I might expect an abstract base class or formal interface.

Part of the difference in expectation here is that Julia does not implement an object oriented programming paradigm. We have a multiple dispatch paradigm which shifts the conceptual emphasis to verbs (i.e. iterate) rather than nouns (i.e. Iterator). The approach here is in fact orthogonal and that is reflected in the documentation.

I think part of your frustration originates from having particular experience with other languages and expecting Julia to be the same. I work in an environment that uses both Java and Python heavily among other languages. While I appreciate that JavaDoc follows a particularly pattern that I familiar with, I see my junior colleagues struggle with this pattern. Thus, I’m not particularly convinced that Java is the best example here. It’s helpful to you and I because we are familiar with the language, but others who are not familiar with Java would struggle with it just as you are struggling with the Julia documentation.

Perhaps one thing to take away from JavaDoc is having some regular structure to Julia docstrings. What pattern should arguments follow in implementing methods? What return type is expected? Is this function part of a larger interface?

From your elaboration, I see that your main interface here is to use search. Alternatives might be using the Table of Contents or using a LLM. This is helpful and thus I can formulate some concrete action items.

  1. We have a module called Base.Iterators that people may land upon when searching for items. Modules should contain some documentation and link to the Iteration Interface section: julia/doc/src/base/iterators.md at master · JuliaLang/julia · GitHub
  2. While Iteration Interface is referred to in the section introduction above where Base.iterate documented, it is not referred to within the docstring for iterate itself. We should refer to interfaces in the docstrings of functions belonging to those interfaces: julia/base/essentials.jl at 966d0af0fdffc727eb240e2e4c908fdd46697e57 · JuliaLang/julia · GitHub
  3. Search should emphasize headings in the Essentials and Manual sections. Search should also expand nouns to their analogous verbs (e.g. “Iterator” => “iterate” or “iteration”).
  4. We may need an explicit comparison between object orientation and multiple dispatch in the manual perhaps with an emphasis on interfaces.

Thank you. Let me see if we can make progress on some of these action items.

5 Likes
Adopt the SciML Style Guide for Julia Base and Standard Library?
24

Maybe, part of these frustrations are simply because of how search works? I regularly find search in julia docs (and other Documenter docs)… underwhelming.

4 Likes
25

(and your welcome!)

I think part of your frustration originates from having particular experience with other languages and expecting Julia to be the same.

You are reflecting what others have expressed about the distinction between JL and other languages, but I think this is a distraction. Even if I were trying to look up something C++ or Java I rarely care about the object or interface. Moreover, my general experience with looking up stuff for other languages, including Python and JavaScript, blah blah blah is that the search experience renders discoverable education about these languages.

Perhaps one thing to take away from JavaDoc is having some regular structure to Julia docstrings. What pattern should arguments follow in implementing methods? What return type is expected? Is this function part of a larger interface?

This is (or was) the essence of the Java philosophy

The value of the javadoc system was to make describing the contract easy and automating links to constructs fulfilling such contracts. Endlessly helpful for the years I spent in that domain.

Design by Contract is meaningful regardless of OO or functional etc.

==
Finding stuff that I need:

  1. I am not going to use an LLM. It’s bad form to assume people will. DuckDuckGo’s search assistant offers generally functional outcomes, but not refined ones. LLMs scrape substack. Much of what is referred to is years old. (see below)
  2. TOC is good for a discourse on a topic, not so good for “how do I get X done” queries
  3. your other suggestions are quite good

Let me continue with use case as an individual with glancing familiarity to JL

Let’s assume I have a task and search based on how I think about it

find element in a collection julia
vs
find elements in a collection julia
vs
find an element in a collection julia
vs
find if an element in a collection julia

(try them and see what hits come up for you)

The range of options for finding stuff in collections is a bit more sophisticated than just findall() or in().

doesn’t link to DataFrames or structs, and DataFramesMeta as an example. It really should. But I don’t know how practical any of that would be. javadoc does it by traversing a class’ imports and package.

In summary:

The biggest obstacle is that the theoretic and generic high order modules and interfaces dominate search results both from search engines and in the documentation system.

LLM draws from answers from years ago. Traditional search suffers the same problem.

Julia’s documentation system doesn’t flow between theoretic and concrete. This is a road block to discovery.

26

I think one of the problems with the Julia documentation is that dozens of docstrings are crammed onto the same page, which incentivizes short docstrings. Each docstring should have its own page.

1 Like
27

PHP has this convention where a function has its own page:

I’m not sure I would want each docstring to have its own page. It might be useful for each function to have a page with a summary of methods grouped by module though.