Deploy Literate non-html files with Documenter

General Usage
, , ,
1

I’m using Literate.jl to convert example scripts to markdown (md) and Jupyter Notebook (ipynb) for the documentation. The scripts are in examples, and I have a script in docs/generate.jl that gets run as part of docs/make.jl to convert the examples/*.jl files into md and ipynb files in the docs/examples/ folder. I’ve copied this workflow from Ferrite.jl. When I execute docs/make.jl, I see that (as expected), the output folder docs/build/examples contains the html file rendered from the md, as well as the ipynb file for each example.

That is, docs/build contains

.
├── [...]
├── assets
│   └── [...]
├── examples
│   ├── index.html
│   ├── simple_state_to_state.html
│   ├── simple_state_to_state.ipynb
│   ├── state_to_state_rwa.html
│   └──state_to_state_rwa.ipynb
├── index.html
└── [...]

The output from my Github Pipeline seems to indicate that these files also get properly generated in the CI build folder.

However, the .ipynb files do not get copied over to the gh-pages branch during the CI deployment. The gh-pages branch contains the following files after deployment:

.
├── dev
│   ├── api
│   │   └── index.html
│   ├── assets
│   │   └── [...]
│   ├── examples
│   │   ├── index.html
│   │   ├── simple_state_to_state
│   │   │   └── index.html
│   │   └── state_to_state_rwa
│   │       └── index.html
│   ├── index.html
│   └── [...]
└── versions.js

I’ve also tried this with the prettyurl option for Documenter.jl set to false, which doesn’t help (it just replaces, e.g., simple_state_to_state/index.html with simple_state_to_state.html)

I haven’t found any documentation for what exactly Documenter.jl does during the deployment to gh-pages. Clearly, it does not simply copy over the entire docs/build folder, as I would have expected. Also, the deployment of the .ipynb files seems to work for Ferrite.jl as well as for Literate.jl itself. I don’t see that I’m doing anything different from those two packages. Does anybody have any experience or ideas on how to get Documenter.jl to deploy .ipynb files from docs/build to gh-pages?

1 Like
2

Could you run tree between makedocs and deploydocs to make sure that src/ and build/ are what they should be? E.g. with this snippet:

cd(() -> run(`tree`), @__DIR__)

As another breadcrumb, the copying by Documenter happens in gitrm_copy. You could add some debug statements there to see what’s happening.

2 Likes
3

Thanks, that was extremely helpful!

Adding some of my own @debug statements to Documenter.jl shows that it really just copies the build folder to the destination directory on gh-pages. Running tree also shows that the ipynb files do in fact get copied correctly.

The problem is in commiting the changes to the gh-pages branch. What is happening is that there is a stray .gitignore file in docs/build/examples that excludes ipynb files. Thus, when deploydocs runs git add -A . to deploy to the gh-pages branch, the files simply are not added.

The stray .gitignore file is originally from docs/src/examples, and makedocs simply copies it over. That seems like it might be a bug. The reason I have docs/src/examples/.gitignore is that I’m generating they .ipynb files into that folder before running makedocs, but obviously they shouldn’t be committed. I’m not sure I would expect makedocs to copy .gitignore files to the build folder though: I can’t think of a situation where I would want that to happen, but maybe there is one. What do you think? Should I raise an issue for that?

In any case, manually removing the stray .gitignore file between makedocs and deploydocs completely solves the problem.

Thanks you for your help!

2 Likes
4

Thanks @goerz this issue plagued me for ages. Could we maybe add it to the Ltierate.jl docs?

Apologies if it’s already there and I just missed it…