Quarto + Typst (+ tdf ) save your eyes from reading documents

Do you know the Riemann zeta function?

\zeta(s) = \sum_{n=1}^\infty \frac{1}{n^s}

Especially, for s=2 we have \zeta(2) = \pi^2/6. The \zeta(s) is implemented in SpecialFunctions.jl. Indeed,

julia> using SpecialFunctions
julia> ? # switch to help mode.
help?> zeta
search: zeta Meta eta beta lbeta ncbeta let get! detach zero

  zeta(s, z)

  Generalized zeta function defined by

  \zeta(s, z)=\sum_{k=0}^\infty \frac{1}{((k+z)^2)^{s/2}},

  where any term with k+z=0 is excluded. For \Re z > 0,
  this definition is equivalent to the Hurwitz zeta
  function \sum_{k=0}^\infty (k+z)^{-s}.

  The Riemann zeta function is recovered as
  \zeta(s)=\zeta(s,1).

  External links: Riemann zeta function
  (https://en.wikipedia.org/wiki/Riemann_zeta_function),
  Hurwitz zeta function
  (https://en.wikipedia.org/wiki/Hurwitz_zeta_function)

  ─────────────────────────────────────────────────────────

  zeta(s)

  Riemann zeta function

  \zeta(s)=\sum_{n=1}^\infty \frac{1}{n^s}\quad\text{for}\quad s\in\mathbb{C}.

  External links: Wikipedia
  (https://en.wikipedia.org/wiki/Riemann_zeta_function)

julia>

Yeah, the LaTeX syntax

  \zeta(s)=\sum_{n=1}^\infty \frac{1}{n^s}\quad\text{for}\quad s\in\mathbb{C}.

corresponds to the formula.

\zeta(s) = \sum_{n=1}^\infty \frac{1}{n^s}

Since many people in the scientific domain have a LaTeX parser in their eyes or their brains, “we don’t say we can’t understand it,” but it is less readable.

If we want to see these docstrings more intuitively, we need to navigate the following page, which requires an internet connection.

https://specialfunctions.juliamath.org/stable/functions_list/#SpecialFunctions.zeta-Tuple{Number}

Is there any more intuitive way to visualize docstrings, including equations written in LaTeX? Let’s find out.


Quarto + Typst → PDF

Let’s consider the following script:

import REPL # This requires to allow us to collect all docstring related to zeta
using Markdown

using SpecialFunctions
using quarto_jll

run(`rm -f example.pdf example.qmd`)

frontmatter = """
---
format:
  typst:
    keep-typ: false
    margin:
      x: 1cm
      y: 1cm
    mainfont: "Hiragino Mincho ProN"
    fontsize: "24pt"
---
"""

md = Markdown.MD(@doc zeta).content[begin]

io = IOBuffer()
println(io, frontmatter)

for t in md.content
    println(io, t)
end

str = String(take!(io))
output_text = replace(str, r"```math\s*(.*?)\s*```"m => s"$$\1$$", r"``(.*?)``" => s"$\1$")
write("example.qmd", output_text)

run(`$(quarto()) render example.qmd --to typst`)

Here, we used quarto_jll.quarto(), which provides the quarto command. Quarto is an open-source scientific and technical publishing system. https://quarto.org/

The Typst allows to generate PDF fast compared to pandoc.

Bang!!! :tada: We could render a PDF containing Julia docstrings that explain the zeta function. Are you satisfied?


PDF + tdf + Terminal (supports Sixel Graphics) → :smile:

We are greedy. Do we want to display the generated PDF nicely on your terminal? So, how do we do that?

Recently, GitHub - itsjunetime/tdf: A tui-based PDF viewer is released. A tui-based PDF viewer written in Rust. Le’ts compile it using cargo:

git clone https://github.com/itsjunetime/tdf.git
cd tdf
cargo build --release

These commands will create a binary target/release/tdf. The tdf binary is self-contained. We can move or install it to your preferred directory. For instance, since I installed Julia using juliaup, the environment variable PATH should contain ~/.juliaup/bin.

So, I decided to run the mv command:

mv target/release/tdf ~/.juliaup/bin

(Maybe we should have run cargo insatll --git https://github.com/itsjunetime/tdf.git ???)

$ tdf --help
Usage:  <file> [-r <r_to_l>] [-m <max_wide>] [-h]
Arguments:
  <file>               PDF file to read

Options:
  -r, --r-to-l <r_to_l> Display the pdf with the pages starting at the right hand size and moving left and
adjust input keys to match
  -m, --max-wide <max_wide> The maximum number of pages to display together, horizontally, at a time
  -h, --help           Prints help

Commands:
  help                 Print this message or the help of the given subcommand(s)

Let’s run the following command:

$ tdf example.pdf

If your terminal supports sixel graphics, the tdf will convert the PDF as an image and display it on your terminal nicely :sunglasses: .

Conclusion

We used Julia, and the Rust ecosystem to enable intuitive visualisation.

import REPL # allow us collect all docstring related to zeta
using Markdown

using SpecialFunctions
using quarto_jll

run(`rm -f example.pdf example.qmd`)

frontmatter = """
---
format:
  typst:
    keep-typ: false
    margin:
      x: 1cm
      y: 1cm
    mainfont: "Hiragino Mincho ProN"
    fontsize: "24pt"
---
"""

md = Markdown.MD(@doc zeta).content[begin]

io = IOBuffer()
println(io, frontmatter)

for t in md.content
    println(io, t)
end

str = String(take!(io))
output_text = replace(str, r"```math\s*(.*?)\s*```"m => s"$$\1$$", r"``(.*?)``" => s"$\1$")
write("example.qmd", output_text)

run(`$(quarto()) render example.qmd --to typst`)

#=
git clone https://github.com/itsjunetime/tdf.git
cd tdf
cargo build --release
mv target/release/tdf ~/.juliaup/bin
=#

run(`tdf example.pdf`)

Your feedback is highly appreciated.

Have a nice Julian day!

26 Likes

Julia encourages the use of Unicode, falling back on LaTeX constructs only when absolutely necessary. See point 7 in Writing Documentation. The resulting mix of Unicode and LaTeX tends to be extremely readable as plain text.

I think you underestimate how much variation there is in the contexts where docstrings enter. Keep it simple! (Not that Quarto isn’t a nice piece of software)

3 Likes

See point 7 in Writing Documentation

I know this feature.

I think you underestimate how much variation there is in the contexts where docstrings enter. Keep it simple!

Could you explain more specific?

I agree that writing symbols in Unicode improves readability. Not all docstrings in packages that have existed for a long time actively use Unicode. That’s why I posted this post … :thinking:

Other than SpecialFunctions.jl, there are other packages that make heavy use of LaTeX syntax, such as Distributions.jl.

The attahced image in the L.H.S is the output of docstring of Distributions.Gamma.

The R.H.S is generated by my script as shown above in my post.

At least the method I have proposed does not lose readability for the parts written in Unicode and improves readability for the parts written in LaTeX.

Keep It Simple! is easy to say. I think we can have a more constructive discussion by giving concrete examples … I have already provided at least two examples …

2 Likes

How about this?
The author may include both unicode and typst (or latex for antiquarians) in the doc string, labeled with some kind of markup. The command you use to render it will choose what to do when it encounters one or the other or both. For example the REPL help interface would continue to prefer unicode.
This requires more labor.

Having readable math (you can’t get this just through unicode) in the terminal would be great.

7 Likes

You could do it like following

"""
   function foo(x)

general docs
$(isdefined(Main, :MathRenderingPackage) ? 
"here latexified 

or typst docs"
:
"here unicode docs"
)
"""
function foo(x)
    return x
end

Note that Typst works as a backend for generating PDF. It’s similar to GR.jl inside Plots.jl to plot something. We do not have to learn new syntax related to Typst. The auto generated example.qmd is just a Markdown file with an optional fromtmatter:

---
format:
  typst:
    keep-typ: false
    margin:
      x: 1cm
      y: 1cm
    mainfont: "Hiragino Mincho ProN"
    fontsize: "24pt"
---

```
Gamma(α,θ)
```

The *Gamma distribution* with shape parameter `α` and scale `θ` has probability density function

$$
f(x; \alpha, \theta) = \frac{x^{\alpha-1} e^{-x/\theta}}{\Gamma(\alpha) \theta^\alpha},
\quad x > 0
$$

```julia
Gamma()          # Gamma distribution with unit shape and unit scale, i.e. Gamma(1, 1)
Gamma(α)         # Gamma distribution with shape α and unit scale, i.e. Gamma(α, 1)
Gamma(α, θ)      # Gamma distribution with shape α and scale θ

params(d)        # Get the parameters, i.e. (α, θ)
shape(d)         # Get the shape parameter, i.e. α
scale(d)         # Get the scale parameter, i.e. θ
```

External links

  * [Gamma distribution on Wikipedia](http://en.wikipedia.org/wiki/Gamma_distribution)

I should have added more comments, sorry for the confusion everyone :bowing_man:

import REPL # This allows us to collect all docstring related to zeta
md = Markdown.MD(@doc Gamma).content[begin]

# Dump docstrings as Markdown
begin
  io = IOBuffer()
  println(io, frontmatter)

  for t in md.content
      println(io, t)
  end

  str = String(take!(io))
  output_text = replace(str, r"```math\s*(.*?)\s*```"m => s"$$\1$$", r"``(.*?)``" => s"$\1$")
  write("sample.qmd", output_text)
end

# Geenerate PDF using Quarto
# The `--to typst` tells quarto to make PDF using typst.
# We do not have to learn about typst.
# We can ommit `--to typst`
run(`$(quarto()) render sample.qmd --to typst`)
2 Likes

I think we can make a solid improvement here by introducing rough Unicode LaTeX rendering:

ζ(s) = ∑ₙ₌₁^∞ 1 / nˢ  for  s ∈ ℂ.

I think we can make a solid improvement here

How do we make it in JuliaLang? Could you share your code?

Typst can also render directly to png in which case you don’t need tdf. Not sure how to invoke quarto to get png though, maybe you’d need to render to a typst file first and invoke typst yourself.

One could also take each math bit and render it to a small, tightly cropped png directly with Typst, then place that png in the normal printed text using sixels. Then you don’t need to typeset the whole text.

Good to know, I will check this. Thank you!!!

using TerminalPager
using Sixel
using TerminalGat
using FileIO, ImageIO
using Markdown
using SpecialFunctions
using quarto_jll

run(`rm -f sample.pdf sample.qmd`)

frontmatter = """
---
format:
  typst:
    keep-typ: true
    margin:
      x: 2cm
      y: 2cm
    mainfont: "Hiragino Mincho ProN"
    fontsize: "24pt"
---
"""

import REPL # allow us collect all docstring related to zeta
md = Markdown.MD(@doc zeta).content[begin]

io = IOBuffer()
println(io, frontmatter)

for t in md.content
    println(io, t)
end

str = String(take!(io))
output_text = replace(str, r"```math\s*(.*?)\s*```"m => s"$$\1$$", r"``(.*?)``" => s"$\1$")
write("sample.qmd", output_text)

run(`$(quarto()) render sample.qmd --to typst`)
@assert isfile("sample.typ")
run(`$(quarto()) typst compile sample.typ sample_\{n\}.png`)

imgs = filter(readdir(".")) do f
  pattern = r"^sample_[0-9]+\.png$"
  m = match(pattern, f)
  !isnothing(m)
end .|> load

for c in Iterators.partition(imgs, 2)
  Sixel.sixel_encode(hcat(c...))
end

Since tdf implements pager, I still want the tdf command.

Or we could write it directly with typst syntax as

ζ(s) = ∑_(n=1)^∞ 1 / n^s " for " s ∈ ℂ

(Yes, typst renders that exactly like the latex above.)

If you want to see it yourself, run

using Typstry
render(typst"""$ζ(s) = ∑_(n=1)^∞ 1 / n^s " for " s ∈ ℂ$""");
8 Likes

Sure, but Julia Markdown explicitly states that the contents of ``...`` is LaTeX, and I don’t think you’d have much luck (at this stage at least) campaining for a new Typst-specific syntax extension.

If I recall correctly, MakieTeX.jl does something similar!

Yeah, I mostly wanted to point out that there’s something roundabout about asking quarto to call pandoc to convert latex to typst, so quarto then can call typst to render it to png. Instead it could have been much more readable in the source and required far less tooling to make a png or pdf.

1 Like

Right. There’s an alternative here though, that’s LaTeX → Unicode.

1 Like