[documenter] From Notebook to Markdown with docstrings?

General Usage
1

I am a big fan of documenter.jl! It lets you create beautiful documentations whilst still having full control over its code. Currently, I use the @example syntax extensively, but I would like to docstrings as they provide more rigorous testing. I just so happens that I like to develop examples with notebooks.

So: Is there an automated way to get from Jupyter notebooks to markdown files that contain docstrings? I have read a little bit about literate.jl which goes in the direction.

Here’s an example. Suppose I have written a notebook that looks as follows.

[begin]

56
562320×1256 167 KB

[end]

Where I want to end up is :

[begin]

Computing the 2-norm of a vector

We define a vector x

julia> using LinearAlgebra

julia> x = collect(1:10)
10-element Array{Int64,1}:
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10

and compute its norm via

julia> norm(x)
19.621416870348583

which is equivalent to

julia> isequal(norm(x),sqrt(sum(e^2 for e in x)))
true

[end]

I have written a file that converts a Jupyter notebook to a markdown with @example blocks that documenter.jl provides. The output of this conversion looks like this

[begin]

using LinearAlgebra
x = collect(1:10)
norm(x)
isequal(norm(x),sqrt(sum(e^2 for e in x)))

Computing the 2-norm of a vector

We define a vector x

using LinearAlgebra
x = collect(1:10)

and compute its norm via

norm(x)

which is equivalent to

isequal(norm(x),sqrt(sum(e^2 for e in x)))

[end]

Thanks!

1 Like
2

Let me add a question related to documenter.jl and literate.jl: the default option in literate.jl is to generate @example blocks, see here. Is there a way to set an option such that it creates docstrings?

In general I have nothing against @example blocks, but it’s written in the documentation of documenter.jl that

It’s recommended that as many of a package’s examples be runnable by Documenter’s doctest.

Perhaps @fredrikekre can help?

Thank you :slight_smile:

3

I don’t really understand what you are trying to do. What do you mean with docstrings? Literate returns markdown files, where you don’t have any docstrings, they are embedded in juila source files. If you want doctests instead of @example you can use the codefence kwarg, but then you also have to include the expected output. In general, there might be better ways to test correctness instead of comparing output with expected output. In JuAFEM we do this for example: https://github.com/KristofferC/JuAFEM.jl/blob/bb73b779b935759db01a37f56dd9c4f47e1d962d/docs/src/examples/heat_equation.jl#L187

1 Like
4

I meant to say doctests, yes. I apologize for the confusion.

I’ll take a look at codefence, but I have the feeling that I’ll just stick to what I have → write Jupyter notebooks and execute my script that creates a markdown file with @example blocks which can then be used with documenter.

5

I’m not sure its exactly what you’re looking for, but I do something similar with the docs for my package CMBLensing.jl.

I write the docs as a Jupyter notebook then convert that (along with the outputs) to markdown and then through Documenter. This took writing a custom .tpl template for jupyter nbconvert, as well as a few other tweaks to make Latex equations and certain outputs look right. The basic result is that I start with a notebook like this and get a doc page that looks like this.

You can check out my the whole source here, the magic is all in make.jl and documenter.tpl, perhaps something there is useful. I personally don’t, but you could imagine automating evaluation of the Jupyter notebook (which nbconvert has support for), and then the whole thing put together sounds alot like what you’re describing.

2 Likes
6

Nice! I didn’t know about the template functionality with nbconvert.

What you have is pretty much what I have at the moment (except that your solution is cleaner): I go from here to there. The way I do it is by calling a conversion file.

I guess I’ll settle with what I have :wink:

Can you, @marius311, explain how this function works

1 Like
7

Yea, that’s overwriting this function:

so you can see its just adding the .language thing to the <pre> element as well, and that’s so that then I can select on it with CSS here:

which is how I got the output cells to be white background vs. the gray background for the input cells. Note that for this, I also need my custom template to convert the output cells from the notebook to markdown code cells with language output, which happens here:

1 Like
8

Cool, that’s a nice idea.

9

An update for people stumbling onto this older post, if your aim is to have one set of documents for updating that can be used as both Jupyter notebooks and source for documentation then Literate.jl is probably the way to go, as mentioned by @fredrikekre , at least that is the solution we are likely to use moving forward. It’s really easy to use, and there’s no use case I can think of it doesn’t cover.