[ANN] Native Julia engine for quarto using QuartoNotebookRunner.jl

Package Announcements
1

The dev team at PumasAI is excited to announce that, after months of work, the quarto project has accepted PR #8645 which adds a new, native Julia engine based on PumasAI’s QuartoNotebookRunner.jl package.
The engine is now available in prereleases Release v1.5.29 · quarto-dev/quarto-cli · GitHub until the release of quarto’s next minor version 1.5.

Quarto is the successor to RMarkdown, and a widely used document authoring and publishing system based on Pandoc.

So far, Julia has been supported in quarto via an IJulia Jupyter kernel, which has a couple drawbacks:
In addition to Julia, it requires a working Python and Jupyter installation, the correct configuration of the Julia kernel in this setup, and the availability of IJulia in the main Julia environment.
Many users have been facing difficulties getting this system to work reliably due to its many moving parts, which in turn hinders adoption.

With our new engine, all that’s needed to get started with Julia and quarto is a working installation of each of those tools.
At first render of a qmd file with the engine: julia setting, quarto will automatically install QuartoNotebookRunner into a separate environment, resulting in a frictionless user experience.

With quarto’s RMarkdown legacy, the R engine is currently the most featureful.
With a native Julia engine, we now have more freedom to shape how exactly qmd files are rendered, and can potentially add features currently only available to R users.

All in all, with this work we are striving for a first-class Julia experience with quarto and we hope it will turn out to be useful for you all.

118 Likes
2

Hi,
this is super great news, since currently my documenter CI / runs have a Python dependency, which I would get rid of with this.

Just to clarify (since I could not find instructions on the homepage – yet?) I would take my _quarto.yml configuration and replace

jupyter: julia-1.10

with

engine: julia

?

Because changing that does not seem to have en effect, the quarto render myFile.qmd still fires up a Jupyter (and temporary ipynb file).

3

The engine will soon be available in pre-releases until the release of quarto’s next minor version 1.5.

4

Thanks for the info; I did not see this. Hm, sad, I was super eager to try this out, because my current Documenter.jl workflow is a bit of a chaotic mess due to python dependencies for running quarto.
And on the other hand the post here read like “just add engine: Julia and it works” which read for me – that it is released and not planned to be released somewhen in summer, but I know see I did not read the post careful enough.

5

The prereleases are made every week or so, one with this change included should drop soon (Native Julia engine based on QuartoNotebookRunner.jl by jkrumbiegel · Pull Request #8645 · quarto-dev/quarto-cli · GitHub) and you should be able to use that in CI. The actual minor releases are on a slower schedule, that’s true.

I’ve highlighted the prerelease part now :slight_smile:

1 Like
6

Yes, sorry for not seeing the “on pre-release soon”.
Currently I fear it might take me too much time to convince my brew (local installation) and the CI to use a pre-release (at least I fear that might take me quite some time). So I will probably juts wait. And don’t get me wrong, I subscribed to the PR already when I saw it to keep that in sight – I am really happy this is happening and look forward to trying it :slight_smile:

7

If anyone wants to try before the quarto pre-release happens:

Install quarto with

git clone https://github.com/quarto-dev/quarto-cli
cd quarto-cli
./configure.sh
# In my case I had to use sudo ./configure.sh

Test quarto version:

quarto --version 
# expected output: 99.9.9 
# you can also run `quarto check` to make sure all is setup correctly

If you’re getting something else, it’s your other quarto installation.

I added a simple alias in my .zshrc (depending on your shell):
alias quarto='/Users/USERNAME/.local/bin/quarto'

My MWE quarto - notice the engine: julia in YAML header
You need to ]add DataFrames, StatsPlots in the environment you’re in

Create file demo.qmd with the following contents:

---
title: "StatsPlots Demo"
author: "xxx"
date: "1/1/1900"
format:
  html:
    code-fold: true
engine: julia
---

# Header 1
```{julia}
using DataFrames, StatsPlots
```

# Header 2
```{julia}
df = DataFrame(a = 1:10, b = 10 .* rand(10), c = 10 .* rand(10))
@df df plot(:a, [:b :c], colour = [:red :blue])
```

Second plot:
```{julia}
@df df scatter(:a, :b, markersize = 4 .* log.(:c .+ 0.1))
```

And then run your standard quarto render, eg,
quarto render demo.qmd --to html

12 Likes
8

To add to that, since it’s not easily visible in any docs currently, note that you can control the flags passed to workers that run notebooks via the julia.exeflags key in a notebook’s frontmatter, so you can pick the Project.toml used via that rather than having to use the global environment. See new feature: pure-Julia engine by cscheid · Pull Request #1071 · quarto-dev/quarto-web · GitHub I’m sure their docs site will update with those details sometime soon.

1 Like
9

An additional comment: By default, QuartoNotebookRunner starts the Julia process that the notebook is executed in with --project=@. (Use `--project=@.` as default environment by devmotion · Pull Request #73 · PumasAI/QuartoNotebookRunner.jl · GitHub), so if you put e.g. a Project.toml file next to the notebook it will be picked up automatically. This is the same behaviour as the default IJulia kernel. You can specify some (other) project environment by adding "--project=myproject" to the list of julia.exeflags or by setting the "JULIA_PROJECT=myproject" environment variable in the julia.env key.

5 Likes
10

Question: How PumasAI is using Quarto? Did you mean JuliaHub?

11

Currently, all publicly facing tutorials are written with quarto, https://tutorials.pumas.ai/. It is used internally as well for reporting etc.

13 Likes
12

It is always good to have different options to choose from. As I have never used Quarto as a Julia IDE, and I know Pluto quite well, what are the main advantages(disadvantages) of using one or the other? Thanks.

13

Quarto is not an IDE, you write a plain text notebook format with code fences like

```{julia}
1 + 1
```

and then you render the full notebook through an intermediate markdown representation into tons of formats supported via pandoc. Pluto does not have so many output options because it’s not focused so much on static outputs. For Pluto, I would say the interactive experience is the defining factor.

11 Likes
14

It is super cool! I will use it to write docs and research notes once it is available.

2 Likes
15

I would say a large advantage is, that the Quarto notebooks are Markdown with code cells – and that quarto interoperates with pandoc, so you can get the result in your favourite format. For example in markdown to use it in documenter (my use case). If you need an environment, probably the one of the folder is the one to go for

Pluto is the other way around: Code cells where some of them might be markdown. An advantage might be that each Pluto notebook automatically keeps its own environment, automatically, which I love when exchanging code with students for example. A disadvantage is that rendering Pluto to something is not so easy (but that is on het other hand a main feature of Quarto).

1 Like
16

To extend on this, what I would love to have is a Pluto ↔ Quarto “translator”, that is probably a pandoc receipt.
I love to sketch and develop something in Pluto. But for longtime or tutorial work I prefer Quarto – so somewhere in between I have to “switch”.

2 Likes
17

qvm can help with running multiple versions of quarto.

1 Like
18

If anyone is interested in a working example with Quarto and IJulia, including appropriate configuration of GitHub CI tools, take a look here: Geospatial Data Science with Julia

We have many issues with Makie.jl integration. Hopefully the new QuartoNotebookRunner.jl will improve the situation.

2 Likes
19

Worlds are colliding! Will this be the contribution that gets this manager back to programming?

(Not around much any more in the Julia world, but now Head of Developer Relations at Posit)

16 Likes
20

Sounds nice, though I have not yet understood how to install qvm itself (I am just a poor Mac user with only some terminal knowledge and usually struggle a bit with anything beyond brew)

1 Like