Way to add tests that utilize jupyter notebooks?

General Usage
1

Was wondering what’s the best way to:

  • run jupyter notebooks during a test suite
  • check the output similar to repl output

Would be helpful to have a rosetta stone style answer with tests that are doing string asserts on:

  • repl (text i/o)
  • jupyter (json)
2

I would probably turn to something like Selenium: /documentation/. There are some pre-built Docker images which can help with setup: Docker Hub

3
2 Likes
4

Create the notebooks from Literate.jl files or using Weave.jl, so everything is actually just a text file that is easier to handle.

3 Likes
5

My project CMBLensing.jl might be a useful template to follow, which has documention built entirely from Jupyter notebooks which also double as (extra) tests. Basically:

  • I keep notebooks in the repository as Jupytext Markdown (Jupytext is really nice since you can open and save the .md files directly from Jupyterlab completely transparently as if they were notebooks, so that’s how I develop them).
  • I then have a Dockerfile which builds my entire project (some complex-ish dependencies necessitate this), and inside this file I run nbconvert (here) to execute these notebooks. Should any of the cells error (ok-to-error cells can be marked with a cell tag), the Docker build will fail.
  • I have a GitHub Action set up to build this image alongside normal tests on every push, so its easy to keep track of failures.
  • I then feed these executed notebooks which now have outputs into Documenter.jl and this becomes my documentation, as well as pushing the built image to Docker Hub where it is used as a Mybinder base image.

Its not quite everything you’re asking for but maybe useful ideas.

3 Likes
6

unliterate() is useful for converting notebooks to .jl files. Available at Julia Academy

And then Literate.jl to go in the other direction.

@fredrikekre Have you considered adding an unliterate() to your Literate.jl?

2 Likes
7

See also GitHub - oxinabox/ipynb2LiterateJulia: NBConverter stuff to convert Jupyter Notebooks in to Julia Literate.jl files (This is the opposite of Literate.jl)

2 Likes