Documentation needs to be more discoverable and explorable

General Usage
21

Hi,
If anything like this is going to be implemented, I hope it will be on a voluntary basis, not automatically. Packages have often an large network of dependencies (we can see it very well by looking at the number of packages in .julia/packages) and I wouldn’t like to install tons of manuals for packages that I don’t even know what they are, but are because they are dependencies and dependencies of dependencies of others.

22

Yeah, good point… Maybe my idea that the package manager auto downloads docs is a bit extreme, not sure… Maybe only for packages that are added as top-level deps in projects? Or maybe the right approach is to not download automatically at all, but just host them somewhere centrally and then the VS Code extension can download them and cache them on-demand… Not sure.

23

Could we standardize the docs building process?

Based upon the Documenter.jl process, it might be as follows.

  1. There is a docs directory at the project root.
  2. Within the docs directory there is a make.jl along with a Project.toml and Manifest.toml.
  3. Running julia --project make.jl builds HTML based docs in the docs/build directory.

Perhaps this could be extended to also build a tarball. A Github action could then attach the tarball to the release. This imperative process may be too flexible.

It would also be good to have some machine readable file, a JSON file perhaps, to resolve symbols to their relative documentation URLs. I’m guessing this may already exist.

I really do think we should carefully study existing solutions in other languages to see what works and does not work.

The maven javadoc tooling is a bit heavy handed, but it does seem to have been stable and well used over a decade.

https://maven.apache.org/plugins/maven-javadoc-plugin/usage.html

24

I think the most important thing really is that the choice of output format is not part of something like a make.jl. It shouldn’t be the package author that decides what output formats are produced, that should (somehow) be a user choice. The package author should put in all the relevant info on what is part of the docs etc, but then any user should be able to say “hey, compile this into a HMTL/Markdown/PDF/whatever”, without the need to modify the package itself.

3 Likes
25

That kinda sounds like what happens on the JuliaHub package index, which (in my frank opinion) is bit of a disaster. I’m very doubtful you’d be able to standardize a build of the documentation across the entire ecosystem. For what it’s worth, I’ve never had a docs/Project.toml file (much less a docs/Manifest.toml file) in any of my projects – I don’t see the point of keeping track of multiple dev-environments, so I’m using test/Project.toml both for testing and building the documentation.

If you want local/offline indices of all installed packages, and you want the full documentation (including all tutorials/examples/explanations), pull the documentation from that package’s own built (e.g., by cloning their gh-pages). Although there are also plenty of smaller packages which have their entire documentation in the README.

Alternatively, index just the docstrings. I certainly wouldn’t mind being able to search across the docstrings of all installed or maybe even all available packages. Something like perldoc would seem more like a recommendation for module docstrings (which I don’t see a lot of people using). Maybe we should develop some recommendations for sections in module docstrings, and encourage module docstrings more generally (with improvements to @docs/@autodocs in Documenter).

In any case, docstrings (for functions or of modules) are reference material, so they’re not the full story. Alas, they’re probably the most important part of the documentation while writing code, and I would strongly support more tooling to make them more discoverable and explorable.

3 Likes