What’s the current best way of setting up CI with github actions for Jupyter notebooks containing Julia code?
I’ve found some blog posts, github actions and similar, proposing some solutions, e.g. (1), (2), (3), (4), (5), (6), etc…
I’m just having trouble figuring out which one of these I should be using. Some of these discussions or codes are years old, so I don’t know if they’re still maintained. A lot of these discussions are mostly snippets of github actions various people have put together for this, but I’m not sure if these are robust.
The end goal would be to set up a Binder repo with a bunch of examples on how to use my library. I would ideally like to do a bit of CI by having github actions run all my notebooks and make sure they don’t error out.
I’m the author of (2), but I haven’t updated the forem post for a while. You can checkout my current template repository, grab a copy, and try it out.
The workflow currently:
executes both Literate.jl (.jl ) and Jupyter (.ipynb) notebooks in parallel (since GitHub action runners have 4 CPU cores),
caches executed notebooks to save execution time
publishing a jupyterbook website with Binder support
automatic updates GitHub actions (via dependabot) and Julia dependencies (needs a PAT)
Hello Wen-Wei Tseng (apologies, I’m not sure which part is your first name!)
Thanks for your insightful reply, I am reading all this carefully.
It seems like your solution does a tremendous amount of stuff, far beyond what I am asking, so I’m trying to understand some bits.
You say your thing produces a jupyterbook with Binder support. I clicked around on your github link, and found this. I must admit I’ve never used Binder but my intention was to have a jupyterlab session with my notebooks opened so that people can play around with it. Is this not what Binder does? Can I do that from your jupyterbook?
I took a brief look at jupyterbook and Literate.jl, both look slightly tangential to what I’m trying to achieve. Do I have to use those?
Your solution also uses Kodiak, linkcheck.yml, update-manifest.yml. Now I must admit I’m a bit of a beginner with Github Actions, and I’ve previously failed twice to set up Github Branch Protection, which the Kodiak web site says is a requirement. I’ve also previously failed to setup update-manifest.yml. Are these all requirements?
Binder support is disable by default, you can activate it by editing docs/_config.yml
launch_buttons:
binderhub_url: "https://mybinder.org" # The URL of the BinderHub (e.g., https://mybinder.org)
You don’t have to use Literate notebooks (just putting Jupyter notebooks is OK). The Literate notebooks option is just an additional format the workflow supports. The workflow will try to find and execute jl and ipynb files in the docs folder. (You can see the code in run.jl)
Kodiak (.kodiak.toml), linkcheck.yml, and update-manifest.yml are optional. You can delete them if you don’t need them. Kodiak bot is for automerging updates. linkcheck.yml is for checking http links in notebooks. update-manifest.yml is for automatically updating Julia dependencies every week.