Was wondering what’s the best way to:
- run jupyter notebooks during a test suite
- check the output similar to repl output
Would be helpful to have a rosetta stone style answer with tests that are doing string asserts on:
- repl (text i/o)
- jupyter (json)
I would probably turn to something like Selenium: /documentation/. There are some pre-built Docker images which can help with setup: Docker Hub
2 Likes
Create the notebooks from Literate.jl files or using Weave.jl, so everything is actually just a text file that is easier to handle.
3 Likes
My project CMBLensing.jl might be a useful template to follow, which has documention built entirely from Jupyter notebooks which also double as (extra) tests. Basically:
- I keep notebooks in the repository as Jupytext Markdown (Jupytext is really nice since you can open and save the
.mdfiles directly from Jupyterlab completely transparently as if they were notebooks, so that’s how I develop them). - I then have a Dockerfile which builds my entire project (some complex-ish dependencies necessitate this), and inside this file I run
nbconvert(here) to execute these notebooks. Should any of the cells error (ok-to-error cells can be marked with a cell tag), the Docker build will fail. - I have a GitHub Action set up to build this image alongside normal tests on every push, so its easy to keep track of failures.
- I then feed these executed notebooks which now have outputs into Documenter.jl and this becomes my documentation, as well as pushing the built image to Docker Hub where it is used as a Mybinder base image.
Its not quite everything you’re asking for but maybe useful ideas.
3 Likes
unliterate() is useful for converting notebooks to .jl files. Available at Julia Academy
And then Literate.jl to go in the other direction.
@fredrikekre Have you considered adding an unliterate() to your Literate.jl?
2 Likes