Running `makedocs` overwrites REPL docstrings

Tooling
,
1

I’m having a little problem with the internals of Documenter.jl, in the context of a fork of DocumenterCitation I’m working on (QuantumCitations).

It seems that running makedocs overwrites docstrings of the loaded package. I haven’t been able to figure out where in Documenter’s code this is happening or how I might avoid it.

What I’m doing in the plugin is modifying the values of page.mapping, where page is one of the pages in doc.blueprint.pages, and doc is the main Documenter.Documents.Document. Specifically, I’m modifying markdown links like [GoerzQ2022](@cite) in-place to something like [[2]](<link to bibliography>).

The problem is that these replacements end up back in the actual docstrings. For example, in the test-REPL for the QuantumCitations package itself (make devrepl):

julia> using QuantumCitations

help?> QuantumCitations.Example
  Example

  An example object citing Ref. GoerzQ2022 (@cite) with a "References" section in its docstring.

  References
  ≡≡≡≡≡≡≡≡≡≡≡≡

    •  GoerzQ2022 (@cite) Goerz et al. Quantum 6, 871 (2022)

julia> include("docs/make.jl")
Starting makedocs
...

help?> QuantumCitations.Example
  Example

  An example object citing Ref. [2] (../references/#GoerzQ2022) with a "References" section in its docstring.

  References
  ≡≡≡≡≡≡≡≡≡≡≡≡

    •  [2] (../references/#GoerzQ2022) Goerz et al. Quantum 6, 871 (2022)

Note how the original GoerzQ2022 and @cite have been replaced in the REPL docs!

The real problem with this is that it breaks running makedocs a second time in the same REPL (as I like to do while I’m working on the documentation): on the second run, the plugin picks up the modified docstring, and then doesn’t recognize the link as a citation.

So, again:

  • Where in Documenter.jl does this happen?
  • Why does this happen? Is this a bug in Documenter? I feel like doing replacements while rendering the documentation to HTML shouldn’t affect docstrings in the REPL at all.
  • What can I do to get around this? Can I clear the docstring cache after makedocs?
2

This is obviously pretty deep into the Documenter internals, so I’ll tag @mortenpi and @fredrikekre :wink:

3

Yes, in Documenter <= 0.27, this is expected. Documenter just grabs the (mutable) Markdown objects from the docsystem without copying, and for links it applies its transformation by just mutating the Markdown.Link objects. So they also end up mutated in the docsystem.

What I could suggest is that you use Documenter#master for this. There the internal representation of Markdown has been switched over to MarkdownAST.jl , and so we don’t mutate the original docstrings in the docsystem anymore. But, just as a warning, Documenter’s internals have changed quite a bit on master because of this switch.

1 Like
4

Thanks! That sounds good!

I just tried out the master branch of Documenter, and it’s definitely “very breaking”, so I’ll have to have a look at that when I have a good chunk of time available. Is #documentation on Slack the best place to get some help with adapting to the new internals, for questions like “Where did Documents.walk go ?”

Edit: I see that Documents.walk actually goes to the heart of the change to MarkdownAST.jl. I can see how crossref is implemented in the current master, so I can probably figure it out from there… if not, I’ll let you know :wink:

In the meantime, I will probably be able to mitigate the practical effects to get, e.g., servedocs to work in most situations by caching the citations between runs.

Do you have an idea for when you might want to release the Documenter 0.28 with the backend changes?