[ANN] Glossaries.jl

When you are writing documentation for a package, it might happen that you are repeating yourself

  • when a certain parameter appears in several functions
  • when a certain field appears in several structs
  • when keywords are available in several functions
  • when you have certain, e.g., mathematical terms that you want to resuse

With these these, we identified two challenges

  • keeping these unified, for example when you fix a typo (I do a lot of typos), you have to remember to fix them everywhere
  • keeping a consistent format

Starting from ideas when writing the docs from Manopt.jl, we quite often have a Manifold and a cost function to mention as input parameters.

For that we introduce Glossaries.jl!

The package allows to

  • define a glossary (or sub glossaries) to store for example variables, mathematical terms, or any other terms you would like to store.
  • use formatters to print the entries. This way, a glossary of variables can be used to print input arguments, keyword arguments or even fields, if the names are similar there as well.
  • define a glossary of a package, for example Manopt.current_glossary() that allows other packages to access a Glossary of a package as well.

Example

In Manopt.jl the doc string for gradient descent has 4 parameters, manifold, cost, gradient and start point. These appear in a lot of other solvers as well, so they are stored in the Glossary.

These are now interpolated into the doc string as

where _args is a formatter (or “output generator”) from the Glossary to print arguments, see Reference · Glossaries.jl . This yields that all entries mentioned (=input variables here) are printed as

- `name::type`: description

and the manifold :M is for example defined as

A tutorial can be found at Get started with Glossaries.jl · Glossaries.jl.

14 Likes

Neat! There’s a package I wrote a while ago for Emacs/Org that might be of interest to you, solving the same problem in a different context.

In particular, I expect you might be interested in the syntax and behaviour described in https://code.tecosaur.net/tec/org-glossary/src/branch/master/org-glossary.org.

2 Likes

Interesting relation; I never got around using emacs, when I see people using it, it seems a bit magic to me (similar to people using vim), but yes, interesting to know for others then :slight_smile:

It’s scope creep, but I will admit to hoping it would be to you too :slightly_smiling_face:

More specifically, I think you could have it so that glossaries are defined from markdown, and integrate with Documenter.jl to produce similar result as my package.

Yes, one extension could be to load the Glossary from Markdown or YAML. The main reason this is not done for now and also not planned, is, that Glossaries.jl allows for parametrised entries.

For example

is the definition of the sub gradient (one deterministic element of the subdifferential), so related to :grad_f entry from above. f is always the complete cost function within Manopt, but in some cases arguments might have different names as for the difference-of-convex, where we need (from f = g - h) the subdifferential of a part of the cost and hence have

where we can reuse the same doc string but modify variable name and the display of f.

Since functions are a bit hard to store in YAML/Markdown, for now I am not sure how to safe those to file.


An integration to Documenter.jl would be neat, I like plugins there a lot. Sadly, I am merely a user of Documenter and have zero knowledge about the magic internals, so there someone else would have to come by and help, as to
where, when, how, … to do such an integration.

That all sounds very reasonable. I’m guessing the naming ship may have already sailed, but it seems to me that based on the scope you’ve articulated something like DocstringGlossaries might be a better fit? This could just be me, but when I saw “[ANN] Glossaries.jl” I had no idea what context the package operated in.

Yeah that ship has sailed, the package is registered.

We first considered your name, but then felt, since we also make the Glossary (per Module) available to other modules, that the use might even go beyond a usage in doc strings.

edit: We even considered DocumenterGlossaries – but then we really would have needed an integration to Documenter, which is currently not planned (see above, missing knowledge on my side for example).

Mmmm. I see. If you expand the capabilities that would be cool indeed, and justify the name I think :slight_smile:

1 Like

QED. :slight_smile:

3 Likes

haha burn :smiley: Jokes asside, this is really interesting, thanks for sharing!

We do not yet have an extension to discourse, so repetitions in posts of mine might still appear. :wink:

1 Like