[ANN] PlutoStaticHTML.jl

Package Announcements
, ,
1

Sometimes, a project idea sounds nice, but the outcome just doesn’t work. Sometimes, a project sounds nice and the outcome works reasonably well. And sometimes, a project sounds nice and the outcome works better than expected! Hereby, I like to announce PlutoStaticHTML.jl which worked out tremendously well.

This package is a combination of three things:

  1. Super quick development via Pluto.jl.
  2. Super quick website build because we can re-use Pluto.jl’s parallelization.
  3. Super quick websites because the site is static and plain HTML.

Credits to the Pluto developers for 1 and 2! Multithreading is hard.

When is this package useful?

I’m personally using it for my blog. Tutorials and documentation should work too. In my case, I want to have a few dozen blog posts where most of them contain tables, plots and statistical models. These take 2-15 minutes per post to run. For example, there are posts with a Turing model and Random forests.

Thanks to PlutoStaticHTML, it is possible to:

  1. Build the posts in parallel (see the Build time section below).
  2. Develop quickly; no need to manually choose where to store outputs such as plots. Also, it is very helpful to have reactive notebooks.
  3. Have notebooks with embedded Manifest.toml files. This way, it is easy to re-run each notebook every time, but at the same time it is not necessary to update the dependencies of all notebooks at the same time. No need to update the dependencies of a post if almost no one reads it anyway.

Build time

For my blog, the build time is as follows:

System Build time
GitHub Actions without parallelization 1 hour and 10 minutes
GitHub Actions with parallelization (2 cores) 50 minutes
Self-hosted runner with parallelization (4 cores) 21 minutes

How is this different from the Pluto.jl export to HTML button?

Contrary to the built-in Pluto to “HTML” export, web pages generated with this package work without Javascript. Therefore, the pages load faster and are easier to style. If you want, you can style the output with your own CSS.

Threats to this project

The package relies on Pluto internals to work. I’ve asked Fons what he thinks about PlutoStaticHTML (Raw HTML export · fonsp/Pluto.jl · Discussion #1607 · GitHub) and he seems very open to it. (Thanks @fonsp!)

So, the project might need a few rewrites in the future due to Pluto changing internals, but the code is so simple that it should be doable.

Future plans

I really hope that this project can be incorporated into Books.jl at some point. I already think that Books.jl is better than the R and Python alternatives in quite a few ways, but when combined with Pluto.jl, Books.jl could be 10 times better than the competition.

EDIT: I’ve rebuilt the blog a dozen times now and the build is rock solid. Full CPU load visible via htop:

image
image1652×430 127 KB

30 Likes
2

Would love to see the source for your Turing model example (the link at the bottom didn’t work).

1 Like
3

Does it work with Neptune?

I confess that as long as that I’m forced to use begin end blocks to encapsulate multiple commands I will not use Pluto.

1 Like
4

I wish bigger forces would drive Neptune.jl forward.
This variation of Pluto.jl is highly useful for many cases.

2 Likes
5

Thanks! All notebook links are fixed now below the posts. That specific notebook is at https://huijzer.xyz/posts/notebooks/collinear-bayes.jl and also at posts/notebooks · main · Rik / blog · GitLab.

You can try. It’s not a priority for me. I’m very happy with Pluto.jl.

1 Like
6

Nice package. And the output appears after the input, right? That is very much appreciated.

1 Like
7

Yes. That’s right

1 Like
8

Nice work. Also, it found interesting your blog (in particular your examples with MLJ).

1 Like
9

I’ve just released version 3.0.0 of PlutoStaticHTML.jl. Most noteworthy is a rewrite of the API and the adding of caching (https://rikhuijzer.github.io/PlutoStaticHTML.jl/dev/#Caching). To use the cache, put some HTML files inside a dir and point the package to that dir via previous_dir. Since version 3.0.0, the HTML outputs contain the following information

  1. The SHA checksum of the Pluto notebook file (“.jl”) and
  2. The Julia version used to evaluate the notebook

Note that, when using Pluto’s built-in package manager, (1) contains the package manifest, so (1) ensures that the dependencies are the same in the current run and previous run.

When running parallel_build for a second time and with previous_dir set, the build will re-use the outputs from the previous run if (1) and (2) match.

The previous_dir is very flexible. For example, for the notebooks at

  • joinpath(dir, "notebook1.jl")
  • joinpath(dir, "notebook2.jl")

the cache will look for

  • joinpath(previous_dir, "notebook1.html")
  • joinpath(previous_dir, "notebook2.html")

These HTML files can be located inside a gh-pages branch, but can also be a copy of the published HTML pages on a website. To use the website, just download the webpages and put them in previous_dir.

Build time

For my blog containing 12 or so heavy notebooks with plots and Bayesian models, the build time is as follows:

System Build time
GitHub Actions without parallelization 1 hour and 10 minutes
GitHub Actions with parallelization (2 cores) 50 minutes
Self-hosted runner with parallelization (4 cores) 21 minutes
Runner with parallelization and caching* 3 minutes

*assuming that all notebooks are cache-hits.

In effect, caching makes it possible again to build many notebooks on a basic GitHub Runner. 50 minutes was a bit too long to be convenient, but with caching the running time for changing one notebook will be closer to 10-15 minutes again. This is great for building tutorials and blogs.

7 Likes
10

PlutoStaticHTML now also contains a Documenter output format option. This can be used to generate a Markdown file which can be read directly by Documenter. For example, see this page (built via docs/make.jl).

5 Likes