Some thoughts on improving basic function documentation

Yes. Julia should use :: for return types in docstrings. The fact that it uses -> is completely insane.

I distinctly remember getting confused by this when I was first learning Julia. Not only is f(x) -> T invalid Julia syntax to declare return types, but -> is already used in the language, so to learn Julia you need to know:

  1. :: is for types,
  2. -> is for anonymous functions
  3. Actually no, in docstrings -> is for types.

Julia really is the only programming language I’ve learned were I was worse off after reading the documentation.

There’s an issue about this that has been open since 2017 julia#22804, and the response from the core developers can be summed up as: we (existing users) already know, so you (new users) just have to git gud.

8 Likes

It’s not generally possible to document return types for generic functions. What return type should be used in the docstring for sum(itr), not to mention sum(f, itr)? A short and ideally descriptive name seems more appropriate, just like in the argument list.

Not an enforcement, and not necessarily good.
But I think documentation driven development can help.

2 Likes

You can dispatch to a method in help mode with a function call, call signature, or method signature, it’s just not demonstrated in the primary documentation section for it.

julia> begin
       """f(x)"""
       f(x) = 1
       """f(x::Int, y)"""
       f(x::Int, y) = 2
       end
f

help?> f(1)
  f(x)

help?> f(::Int, ::Float64)
  f(x::Int, y)

help?> f(::Int, ::Any)
  f(x::Int, y)

I haven’t found a way to search methods with partial signatures though, best I can do is check for 1 argument type in methodswith and paste 1 of the printed method signatures into help mode.

Of course they don’t, a method signature or function can be compiled in several ways. A call signature is compiled 1 way, but a fixed output type is still not guaranteed (return type instability). Without the output type, there’s very little you can say beyond intentions. Those intentions could include more details about the return value like the type, that would just be a API standard to adhere to, but such irreversible restrictions are carefully considered and aren’t done lightly. For example, it may seem reasonable to say sum returns a Number, but it would’ve prevented addition of arrays or non-numerical types.

I’d like more of these, and also useful functions like in the docstring of AbstractUnitRange: “step size of oneunit(T) with elements of type T”. It shouldn’t replace descriptive text, just additionally tell the reader how to check properties if there is a standard way to do so.

1 Like

Well, if there’s a lot of flexibility with how ? works, I’m extremely happy to find out more about it (although I’d hope it was better documented).

But assuming that’s the case, I think it’s not great that such flexibility isn’t demonstrated in most of the search results when one tries to look for information on help for specific methods (I’ve looked pretty substantially for it over the years, and not found much). If a package induces a new entry in ? for function foo, why can’t I can’t somehow restrict my search to the “new” entry? I WISH I could just do something like `? PACKAGE.foo, but that seems to be a technical challenge from what I’ve read.

Of course they don’t, a method signature or function can be compiled in several ways.

I do think saying “of course” is unnecessarily pessimistic. :slight_smile: I DO understand that multiuple dispatch creates certain technical challenges around e.g., searching for help on specific methods, or finding easy information on outputs.

But the VAST VAST majority of the time, I don’t think most people really need or care about the lowest level of detail for input/output type. I will nearly never care whether a type is Float64, Float32, or Int – for most input. But that doesn’t mean we can’t provice useful information about the “kind” of outputs a function creates.

If a function makes a type MyType, I don’t actually care (mainly) if the documentation talks about MyType{Float64, Int64, Dict} or MyType{Float8, Int32, Dict). I just want to know: What are the “fields” of MyType (at a high level of abstraction)? If your function produces MyType, what are the core details of MyType I need to know to process the output, etc.?

It’s okay if a developer doesn’t give me the lowest level of detail – I mostly don’t want it. As long as they give me “enough” information to easily write my own code (which doesn’t actually require much detail, due to multiple dispatch), I’m happy. :smiley:

It’s true it’s probably not possible to provide the lowest level of detail alogorithmically and automatically. But I almost never NEED the lowest level of detail. I am perfectly happy to rely on developers to make a decision about the most useful level of detail – if an AbstractArray{SubType} is enough, great. If an Array{Float32} is the easiest level to talk about, the documentation can say so.

I don’t think it’s true that Julia packages can’t provide useful information on their output more consistently. And if that IS true (I’m sure it’s not), then Julia is a much much worse language than I thought. :confused:

2 Likes

That does work, just for separate functions belonging to separate modules.

julia> module A
         """A.foo"""
         function foo end
       end
Main.A

julia> module B
         """B.foo"""
         function foo end
       end
Main.B

help?> A.foo
  A.foo

help?> B.foo
  B.foo

If B had imported A.foo and added new methods to it, the function foo still belongs to A, so B.foo is the same exact function as A.foo, it alone cannot narrow down to the methods in a particular module. You can do that with methods.

julia> module A
         """A.foo"""
         function foo() 0 end
       end
Main.A

julia> module B
         import ..A: foo
         """A.foo(::Int) in B"""
         function foo(::Int) 1 end
       end
Main.B

julia> parentmodule(B.foo)
Main.A

help?> A.foo
  A.foo

  ───────────────────────────────────────────────────────────────────────────────────────────────────

  A.foo(::Int) in B

help?> B.foo
  A.foo

  ───────────────────────────────────────────────────────────────────────────────────────────────────

  A.foo(::Int) in B

julia> methods(A.foo, B) # remember, A.foo === B.foo
# 1 method for generic function "foo":
[1] foo(::Int64) in Main.B at REPL[5]:4

help?> A.foo(::Int64)
  A.foo(::Int) in B

I agree with this part. The thing is, what you see now often is the most useful lowest level of detail. The “lower” your level of detail in a method’s docstring, the more you restrict what that method can do in other people’s code. That makes it a lot less extensible, which goes against Julia principles. Julia is a dynamically typed language with generic functions, flexibility is the goal. You’re still free to restrict a method how you like, you can write more methods with more specific input and output types; at an extreme, a type stable method with only concrete type annotations would only have 1 specific return type, you can document that and isolate that docstring if you provide the full method signature in help mode.

2 Likes

I’m another R user, and agree that its formatting scheme is very helpful. Functions get a title, optionally a short description, and optionally a detailed discussion. These things are presented in a standard order, with a standard typographic convention. This is done automatically. Also, the “check” system ensures that all function parameters have documentation entries. It’s really easy to write documentation in R, including nice facilities for cross-linking to other functions, and automated lists of similar functions.

The upshot of all of this is that it can be very quick to find what you need. I don’t have to wonder where something has been said, because things appear in a certain order, with certain formatting cues like indentation. For an unfamiliar function, I read the whole document. For a familiar one, I just skip to the argument list if I’ve forgotten something, or to the examples if I want a starting point.

I wonder whether it might be helpful to set up an optional framework for documentation, so that authors could try it if they felt so inclined, without using this framework be a requirement.

I’ve authored several R packages, and I can report that the work of documenting is made a lot easier by this standardized scheme. It saves a lot of decision-making about what to say and where to say it.

10 Likes

I think you are describing something like this. Roxygen

1 Like

Yes, I use Roxygen2 for this nowadays. This generates latex-style files that are then interpreted by the R help system. You can also write straight to this latex system, but I think that few folks do that, now that Roxygen2 is available.

My guess is that quite a few Julia authors (and those who might generate documentation PRs) are familiar with the oxygen system, from work in other languages. That be advantageous.

Whether oxygen-style or something else, I do think some sort of framework will ease the task of reading documentation … and writing it, too.

2 Likes

Why doesn’t the Julia compiler automatically insert the function’s signature at the beginning of each docstring? It boggles my mind that the Julia documentation suggests:

Always show the signature of a function at the top of the documentation, with a four-space indent so that it is printed as Julia code.

Why is the developer supposed to do this?? The compiler is the perfect tool to automate it:

  1. It knows everything about the function/method being documented: all the types, all the arguments, all the default values - the entire function/method signature.
  2. It’ in the perfect position to correctly format the function signature and insert it into the docstring because it’s responsible for attaching docstrings to functions anyway.

Python does it right:

>>> def func(a: int, b: float) -> str:
...  "Do important calculation"
...  return a - b
... 
>>> help(func)
Help on function func in module __main__:

func(a: int, b: float) -> str
    Do important calculation

As expected, the docstring begins with the function signature, obviously, because I need to know how to call the function properly!


Now look at Julia:

julia> function func(a::Int, b::Float64)::String
        a - b
       end
func (generic function with 1 method)

help?> func
  No documentation found.

  func is a Function.

  # 1 method for generic function "func" from Main:
   [1] func(a::Int64, b::Float64)
       @ REPL[4]:1

Huh, so the compiler already knows how to insert function signatures into docstrings! However, it didn’t include the return type (why??), even though I explicitly specified it in my code.

Now add a docstring:

julia> "Unhelpful docstring" function foo() 5 end
foo

help?> foo
search: foo floor fourthroot pointer_from_objref OverflowError RoundFromZero

  Unhelpful docstring

Yep, the docstring is unhelpful, and Julia chose to remove the automatically generated function signature from the output, which doesn’t help at all.


Now there’s the great DocStringExtensions.jl package which inserts function signatures and type signatures into your docstrings:

julia> using DocStringExtensions

julia> "$TYPEDSIGNATURES" function bar(a::Int, b=4; c::Bool=true) 5 end
bar

help?> bar
search: bar baremodule SubArray GlobalRef clipboard BitArray backtrace BitMatrix

  bar(a::Int64; ...) -> Int64
  bar(a::Int64, b; c) -> Int64

This is much better, but:

  1. -> Int64 is not how Julia specifies return types. Why is the syntax in the docs different from the syntax of the programming language the documentation describes?
  2. The second signature doesn’t show the arguments’ default values. How can I find these default values?
  3. It also doesn’t show types of keyword arguments, even though my code said c::Bool=true.

I’d prefer these signatures instead:

bar(a::Int64; ...)::Int64
bar(a::Int64, b=4; c::Bool=true)::Int64

Basically, just copy the signature from my code.


As an example of good documentation formatting, I think the Wolfram language does it well. Take a look at this: Plot: Visualize or graph a function—Wolfram Documentation - it’s beautiful, it shows “function signatures” and examples, it tries to link to other relevant documentation pages and each page is structured in the same way:

  1. Initial blue box which shows “function signatures” and very short descriptions of what each signature does.
  2. “Details and Options” is exactly what it says it is.
  3. “Examples” provides concise usage examples
    1. “Basic examples” is one of the first things you see on each documentation page. It gets straight to the point and shows what the function does.
    2. “Scope” goes more in-depth
    3. “Options” discusses the function’s options and provides examples.
    4. “Applications” shows what you can do using this function, what kind of problems you can solve with it.
    5. “Properties & Relations” links this function to other functions: “look, this function is actually a special case of this one! If you wanna do this, use another function!” This is Wolfram’s answer to point 3 in the OP.

Compare this to ?CairoMakie.lines:

help?> CairoMakie.lines
  lines(positions)
  lines(x, y)
  lines(x, y, z)

  Creates a connected line plot for each element in (x, y, z), (x, y) or
  positions.

  NaN values are displayed as gaps in the line.

  Attributes
  ==========
...

Now, what kind of things can I pass as positions? A vector of… something? A tuple of vectors? Some kind of custom struct? What can x and y be? Probably vectors, but what else? Can they actually be vectors? What’s z? The docstring doesn’t really explain how to use this function.

11 Likes

Your example is mistaken, help(func) does not directly print the docstring, print(func.__doc__) does, and it demonstrates nothing is automatically inserted into the docstring. help prints more than just the docstring.

What you’re seeing in Julia’s help mode in the absence of docstrings is the default printing of methods(func), which prints the methods’ signatures. It doesn’t include the return type declaration because 1) it’s not a unambiguous concrete return type, just a convert and typeassert (latter of which is usually abbreviated by the :: syntax in right side expressions), and 2) output types are not part of a method signature. DocStringExtensions.jl’s TYPEDSIGNATURES augments the method signatures with an inferred return type, it does not care what the return type declaration is or if it’s even there:

julia> using DocStringExtensions

julia> "$TYPEDSIGNATURES" function bar(a::Int, b=4; c::Bool=true)::Number 5 end
bar

help?> bar
search: bar baremodule SubArray GlobalRef clipboard BitArray backtrace BitMatrix catch_backtrace

  bar(a::Int64) -> Int64
  bar(a::Int64, b; c) -> Int64

See how the output is inferred as -> Int64 specifically even though I manually declared the supertype ::Number? Now if I had to hazard a guess at why this inference isn’t default behavior, it might be because inference using the abstract types in method signatures is generally useless, we care about function call signatures. It’s also customary to write relevant descriptive text about the return type, which is not something type inference can do.

julia> "$TYPEDSIGNATURES" function baz(x) x end
baz

help?> baz
search: baz

  baz(x) -> Any

Looking at it, none of the arguments have descriptions of their types, either, it relies on the examples. If you want a fair comparison, then you should consider Makie’s documentation, not just docstrings. Here’s the one for lines, and I’d say it does the same job Wolfram page did for Plot.

2 Likes

I think Julia should print more than just the docstring as well. That’s the main point of my comment. Docstrings alone can be pretty unhelpful, and a docstring that doesn’t show how the function can be called is basically useless. Automatically including the appropriate method signature(s) should dramatically improve documentation quality. I think it’s very useful to do ?func (which is the equivalent of Python’s help(func)) and see the function’s signature, like in Python.

I don’t think the Wolfram language even has a notion of “types”: instead, it has raw Heads, like lisps (and Julia’s AST).

As for Makie’s documentation, it’s OK, and indeed, comparing Wolfram’s massive documentation to docstrings isn’t really fair. I was just trying to show what could be improved. For example, Makie’s docstrings could’ve benefitted from at least some examples or at least method signatures like lines(x::AbstractVector, y::AbstractVector). Method signatures can in principle be generated automatically, so adding this to Julia can potentially make many docstrings (or outputs of ?func anyway) a lot more informative.

2 Likes

As you already demonstrated, help mode prints methods(func) if it can’t find any docstrings. If there is a docstring, you use that to mindfully specify what to show, including the methods’ signatures plus any -> suffixes. Your suggested automatic alternatives do not improve things.

Let’s look at inserting method signatures into docstrings; ?+ would print 206 of them. Now it instead explains it’s an addition operator, and addition is known to apply to so many numerical and array types, it’s not worth listing them all. Even when there are types to document, it’s often the umbrella abstract types whose exact methods are rarely used fallbacks; the documented behavior would encompass all the more specific methods without their own docstrings, and help mode’s dispatch on any of their calls will find that docstring (one of the many ways it’s not equivalent to Python’s help). Finally, an umbrella type that explains the possible inputs sometimes does not exist; simple example is iterables, which is also why your suggestion of lines(x::AbstractVector, y::AbstractVector) is misleadingly restrictive.

Now let’s look at help mode printing method signatures separately from docstrings instead, and we don’t rehash the verbosity issue. What if I wrote a method signature into the docstring already, one with good detail that can’t be automatically generated like a descriptive -> return suffix or carefully selected umbrella type annotations? Print both the manual and automatic versions? That’s what Python’s help does, I don’t want that.

In [5]: def func(a: int):
   ...:     """func(a: int) -> int"""
   ...:     return a+1
   ...:     

In [6]: help(func)
Help on function func in module __main__:

func(a: int)
    func(a: int) -> int

You picked a really bad example because you can’t put graphs or any images in a docstring. They could’ve put in examples of function calls with no output, but what’s the point, to annoy people so they learn to search for the full documentation with images?

2 Likes

I just tried it and it turns out you can put images in a docstring.

"""
adsf(x) = 1

![](https://global.discourse-cdn.com/julialang/original/2X/1/12829a7ba92b924d4ce81099cbf99785bee9b405.png)
"""
adsf(x) = 1

showing the doc in the vscode side panel displays the image.

1 Like

Well, Python does it right for Python, but not for Julia. The complicating factor is multiple dispatch. Consider this fairly common pattern:

"""
```julia
    solve(problem; method=:auto, kwargs...)
```
solves the given `problem` using `method`.

All other keyword arguments are passed to the underlying solver.
"""
solve(problem; method=:auto) = solve(problem, method)

solve(problem, method::Symbol) = solve(problem, Val(symbol))

solve(problem, method::Val) = error("Invalid method: $method")

"""
```julia
solve(problem; method=:GRAPE, precision=1e-12)
```

uses the GRAPE method to solve `problem` up to the given `precision`.
"""
function solve(problem, method::Val{:GRAPE})
    # ...
end

Note how solve has a “public” method using a method keyword argument, but several “private” methods with a method positional argument that do not (and should not) show up in the docstring.

Also, the docstring for solve(problem, method::Val{:GRAPE}) is “lying”, since it documents the public API, not the private method that actually implements the solver-method.

2 Likes

The problem is also on which level of abstraction the return should or can be specified.
While it might be true that 1 + 2 returns an Int64 this might not be important. Instead, I usually want to know which operations I can assume on the returned value, i.e., when working with matrices and vectors A * x + b I might care for their shapes, but not the actual types, e.g., whether they are dense or sparse or Hermitian or what not.
With the open and non-formal interface this can be challenging in Julia and requires some care and discipline from the programmer. Less generic or statically typed languages are easier to handle in this respect. Personally, I find Haskell very good in this respect as it is always clear which typeclass (interface) is required for each argument including the return type. Further, the type is usually stated as general as possible. In Python and Julia on the other hand, just reading z = x + y or z = x * y does not imply that z is some numeric type, it might also be a string or AlgebraOfGraphics layer etc.

4 Likes

It doesn’t happen if you retrieve a docstring with @doc or print it in help mode, you just see (Image : ). You need something that can handle links and render an image retrieved from one. That is a nice VScode feature but developers can’t assume users are going to use VScode, so their documentation should not (and usually does not) depend on docstrings alone.

I don’t think old-school platform limitations should inhibit documentation. Emacs can show images, I assume many other editors can too. We’ve been through this for unicode - we can do it again for graphics.

1 Like

It’s not an “old-school” issue, it’s a standardization issue. A feature shouldn’t work in one place (display an image from a link in VScode) and be broken in another (print (Image : ) in the REPL’s help mode). It doesn’t seem straightforward to bring the REPL up to par either, terminals in general are intended to display text, not images; after all, terminals did predate RGB image display. Reading through ImageInTerminal.jl and Sixel.jl, RGB pixels can only be displayed overly large if Sixel isn’t supported by the terminal that the REPL is running on. Unfortunately that includes the Windows terminal for now.

It’s already customary to prepare full documents where you format all the texts and images you need, organized together in chapters and next to tutorials. That’s something method docstrings can never do, so as useful as they are for quick interactive information, they should be a stepping stone to the more organized and detailed sources.

2 Likes

In the documentation of the greater than operator >, it can be mentioned that > can be chained like 1 > x > -1.

2 Likes