# Way to add tests that utilize jupyter notebooks?
**URL:** https://discourse.julialang.org/t/way-to-add-tests-that-utilize-jupyter-notebooks/38341
**Category:** General Usage
**Tags:** question
**Created:** [April 27, 2020, 11:11pm UTC](https://discourse.julialang.org/t/way-to-add-tests-that-utilize-jupyter-notebooks/38341 "2020-04-27T23:11:13Z")
**Posts on this page:** 7
**Page:** 1
### Author:  [@djsegal](https://discourse.julialang.org/u/djsegal)
#### Post date: [April 27, 2020, 11:11pm UTC](https://discourse.julialang.org/t/way-to-add-tests-that-utilize-jupyter-notebooks/38341/1 "2020-04-27T23:11:13Z")
Was wondering what’s the best way to:
- run jupyter notebooks during a test suite
- check the output similar to repl output
Would be helpful to have a rosetta stone style answer with tests that are doing string asserts on:
- repl (text i/o)
- jupyter (json)
---
### Author:  [@oliver](https://discourse.julialang.org/u/oliver)
#### Post date: [May 2, 2020, 12:20am UTC](https://discourse.julialang.org/t/way-to-add-tests-that-utilize-jupyter-notebooks/38341/2 "2020-05-02T00:20:12Z")
I would probably turn to something like Selenium: [/documentation/](https://www.selenium.dev/documentation/en/). There are some pre-built Docker images which can help with setup: [Docker Hub](https://hub.docker.com/u/selenium/)
---
### Author:  [@fredrikekre](https://discourse.julialang.org/u/fredrikekre)
#### Post date: [May 2, 2020, 1:26am UTC](https://discourse.julialang.org/t/way-to-add-tests-that-utilize-jupyter-notebooks/38341/3 "2020-05-02T01:26:02Z")
> [@djsegal](#):
>
> run jupyter notebooks during a test suite
> **[GitHub - JuliaInterop/NBInclude.jl: import code from IJulia Jupyter notebooks...](https://github.com/JuliaInterop/NBInclude.jl)**
>
> import code from IJulia Jupyter notebooks into Julia programs - GitHub - JuliaInterop/NBInclude.jl: import code from IJulia Jupyter notebooks into Julia programs
---
### Author:  [@dpsanders](https://discourse.julialang.org/u/dpsanders)
#### Post date: [May 2, 2020, 1:36am UTC](https://discourse.julialang.org/t/way-to-add-tests-that-utilize-jupyter-notebooks/38341/4 "2020-05-02T01:36:57Z")
Create the notebooks from `Literate.jl` files or using `Weave.jl`, so everything is actually just a text file that is easier to handle.
---
### Author:  [@marius311](https://discourse.julialang.org/u/marius311)
#### Post date: [May 2, 2020, 2:22am UTC](https://discourse.julialang.org/t/way-to-add-tests-that-utilize-jupyter-notebooks/38341/5 "2020-05-02T02:22:43Z")
My project [CMBLensing.jl](https://github.com/marius311/CMBLensing.jl) might be a useful template to follow, which has documention built entirely from Jupyter notebooks which also double as (extra) tests. Basically:
- I keep notebooks in the repository as Jupytext Markdown (Jupytext is really nice since you can open and save the `.md` files directly from Jupyterlab completely transparently as if they were notebooks, so that’s how I develop them).
- I then have a Dockerfile which builds my entire project (some complex-ish dependencies necessitate this), and inside this file I run `nbconvert` ([here](https://github.com/marius311/CMBLensing.jl/blob/411c679d1e71a6c3d99a45bdb3d3ab6bdd27c7e7/Dockerfile#L110-L114)) to execute these notebooks. Should any of the cells error (ok-to-error cells can be marked with a cell tag), the Docker build will fail.
- I have a GitHub Action set up to build this image alongside normal tests on every push, so its easy to keep track of failures.
- I then feed these executed notebooks which now have outputs into Documenter.jl and this becomes my documentation, as well as pushing the built image to Docker Hub where it is used as a Mybinder base image.
Its not quite everything you’re asking for but maybe useful ideas.
---
### Author:  [@Paul\_Soderlind](https://discourse.julialang.org/u/Paul_Soderlind)
#### Post date: [May 2, 2020, 6:56am UTC](https://discourse.julialang.org/t/way-to-add-tests-that-utilize-jupyter-notebooks/38341/6 "2020-05-02T06:56:34Z")
`unliterate()` is useful for converting notebooks to .jl files. Available at [Julia Academy](https://github.com/JuliaAcademy/JuliaAcademyMaterials/blob/master/unliterate.jl)
And then `Literate.jl` to go in the other direction.
@fredrikekre Have you considered adding an `unliterate()` to your `Literate.jl`?
---
### Author:  [@fredrikekre](https://discourse.julialang.org/u/fredrikekre)
#### Post date: [May 4, 2020, 9:01am UTC](https://discourse.julialang.org/t/way-to-add-tests-that-utilize-jupyter-notebooks/38341/7 "2020-05-04T09:01:22Z")
> [@Paul\_Soderlind](#):
>
> @fredrikekre Have you considered adding an `unliterate()` to your `Literate.jl` ?
See also [GitHub - oxinabox/ipynb2LiterateJulia: NBConverter stuff to convert Jupyter Notebooks in to Julia Literate.jl files (This is the opposite of Literate.jl)](https://github.com/oxinabox/ipynb2LiterateJulia)