MakieTypstEngine.jl: a proof of concept for rendering typst strings in makie

Package Announcements
, ,
1

During the hackathon at JuliaCon in Paris, @Alseidon and I started playing around with passing typst strings (from typstry.jl) through makie, to a custom wrapper around the typst compiler, and back to makie, essentially trying to create a MathTeXEngine.jl equivalent, but for typst. (Thanks to @Kolaru for talking us through the makie side of things of the whole latex string rendering)

We managed the whole idea working up to a proof of concept state, which I would like to put out here for more people to try out: MakieTypstEngine.jl (be aware that we are compiling our typst wrapper binary on your machine when you load the package, so for it to work you need to have cargo available)

At the moment, the functionality the package provides has fully replaced (the often fiddly) MakieTeX.jl whenever I need to make figures for a presentation or poster that I want to nicely integrate within a typst document.

To register this as a real package, there are a few more things to polish, which are on the one hand about getting rid of the whole type piracy situation as well as not having to build the typst binary on the users machine (probably set up a jll thingy for that), as well as having someone who actually knows something about font rendering look over the way we are currently doing things, and help with making this more stable. (NewComputerModern for some reason is working well on the typst side, but the glyphs indices that we get back are pretty scrambled up)

In addition, the current api to get your fonts from makie to typst and back is working but feels somewhat over-engineered and brittle. Not sure if there is anything that we can do about this, as font resolving seems to be a general problem everywhere…

So with all that warning, I would love if you try it out and tell us what you think.

29 Likes
2

This is cool! I thought about using typst for this kind of thing previously but I kind of stopped when it would have been time to research how to get the glyph and line layout data out of typst into Makie. The json method is interesting, maybe it would even be possible to give your Rust tool a C compatible ABI and load it as a jll such that we can directly read the layout data as an array from Julia. In case the json is a bit too slow or just because it’s cleaner.

We could try opening up the type constraints a bit from the Makie side so that this kind of text thing can actually be cleanly overloaded without type piracy. I know that Anshul has wanted this for a while for his LaTeX stuff.

1 Like
3

Really exciting! Nice to actually get the text layout directly.

1 Like
4

I’d love to give this a whirl, but I’m having some problems getting the tutorial example to work.

First, for some reason I cannot use the font used in the readme example, I get the following error:

font = MakieTypstEngine.MTEFont("TeXGyrePagella")
ERROR: SystemError: opening file "/home/jonas/.julia/packages/MathTeXEngine/8VxpV/assets/fonts/TeXGyrePagellaMTE/TeXGyrePagellaMTE-Regular.otf": No such file or directory
Stacktrace:
  [1] systemerror(p::String, errno::Int32; extrainfo::Nothing)
    @ Base ./error.jl:176
  [2] systemerror
    @ ./error.jl:175 [inlined]
  [3] open(fname::String; lock::Bool, read::Bool, write::Nothing, create::Nothing, truncate::Nothing, append::Nothing)
    @ Base ./iostream.jl:295
  [4] open
    @ ./iostream.jl:277 [inlined]
  [5] open(fname::String, mode::String; lock::Bool)
    @ Base ./iostream.jl:358
  [6] open(fname::String, mode::String)
    @ Base ./iostream.jl:357
  [7] open(::FreeTypeAbstraction.var"#1#2", ::String, ::Vararg{String}; kwargs::@Kwargs{})
    @ Base ./io.jl:408
  [8] open
    @ ./io.jl:407 [inlined]
  [9] newface_mmapped(filepath::String, faceindex::Int64, ftlib::Vector{Ptr{FreeType.FT_LibraryRec_}})
    @ FreeTypeAbstraction ~/.julia/packages/FreeTypeAbstraction/GGCCW/src/types.jl:34
 [10] newface_mmapped
    @ ~/.julia/packages/FreeTypeAbstraction/GGCCW/src/types.jl:34 [inlined]
 [11] FTFont (repeats 2 times)
    @ ~/.julia/packages/FreeTypeAbstraction/GGCCW/src/types.jl:163 [inlined]
 [12] #8
    @ ~/.julia/packages/MathTeXEngine/8VxpV/src/engine/fonts.jl:21 [inlined]
 [13] get!(default::MathTeXEngine.var"#8#9"{String}, h::Dict{String, FreeTypeAbstraction.FTFont}, key::String)
    @ Base ./dict.jl:458
 [14] load_font
    @ ~/.julia/packages/MathTeXEngine/8VxpV/src/engine/fonts.jl:20 [inlined]
 [15] get_font
    @ ~/.julia/packages/MathTeXEngine/8VxpV/src/engine/fonts.jl:224 [inlined]
 [16] MTEFont(name::String, style::Symbol)
    @ MakieTypstEngine ~/.julia/packages/MakieTypstEngine/z5zcV/src/font_resolve.jl:103
 [17] MTEFont(name::String)
    @ MakieTypstEngine ~/.julia/packages/MakieTypstEngine/z5zcV/src/font_resolve.jl:94
 [18] top-level scope
    @ REPL[3]:1

When I go look in that folder, there is clearly a file called “TexGyrePagellaMTE-Regular.otf”, however, running the following command in my Linux shell:

ls /home/jonas/.julia/packages/MathTeXEngine/8VxpV/assets/fonts/TeXGyrePagellaMTE/TeXGyrePagellaMTE-Regular.otf

also reports that the file cannot be found in spite of the fact that running

ls /home/jonas/.julia/packages/MathTeXEngine/8VxpV/assets/fonts/TeXGyrePagellaMTE/

results in this output:

LICENSE                           TexGyrePagellaMTE-Italic.otf
TexGyrePagellaMTE-BoldItalic.otf  TexGyrePagellaMTE-Math.otf
TexGyrePagellaMTE-Bold.otf        TexGyrePagellaMTE-Regular.otf

Is there some weird unicode-look-alike shenanigans going on here?

Second. I could successfully load another font, “LucioleMath”, but this then generates the following error:

Label(fig[1, 2], typst_string, fontsize = 20, tellheight = false)
ERROR: Failed to resolve arg1:
[ComputeEdge] arg1 = compute_identity((linesgments_shifted, ), changed, cached)
  @ /home/jonas/.julia/packages/ComputePipeline/03tW7/src/ComputePipeline.jl:737
[ComputeEdge] linesgments_shifted = (::MapFunctionWrapper(#2928))((linesegments, lineindices, preprojection, model_f32c, positions_transformed_f32c, model_clip_planes, space, ), changed, cached)
  @ unknown method location
[ComputeEdge] glyphcollections, glyphindices, font_per_char, glyph_origins, glyph_extents, text_blocks, text_color, text_rotation, text_scales, text_strokewidth, text_strokecolor, linesegments, linewidths, linecolors, lineindices = #2906((input_text, fontsize, selected_font, align, rotation, justification, lineheight, word_wrap_width, offset, fonts, computed_color, strokecolor, strokewidth, ), changed, cached)
  @ /home/jonas/.julia/packages/Makie/4JW9B/src/basic_recipes/text.jl:333
  with edge inputs:
    input_text = Typstry.TypstStrings.TypstString[Typstry.TypstStrings.TypstString(Typstry.TypstTexts.TypstText("this is an integral:\n\$ integral_0^t sin(x)^2 dif x \$\n"))]
    fontsize = 20.0f0
    selected_font = FTFont (family = TeX Gyre Heros Makie, style = Regular)
    align = (:center, :center)
    rotation = 1.0 + 0.0im + 0.0jm + 0.0km
    justification = :center
    lineheight = 1.0f0
    word_wrap_width = -1.0f0
    offset = Float32[0.0, 0.0, 0.0]
    fonts = Attributes()
    computed_color = RGBA{Float32}(0.0, 0.0, 0.0, 1.0)
    strokecolor = RGBA{Float32}(0.0, 0.0, 0.0, 0.0)
    strokewidth = 0.0f0
Triggered by update of:
  position, text, arg1, fontsize, fonts, font, align, rotation, justification, lineheight, word_wrap_width, offset, fonts, color, colorscale, alpha, colorrange, colorscale, color, colorscale, alpha, colormap, alpha, nan_color, lowclip, colormap, alpha, highclip, colormap, alpha, colormap, alpha, strokecolor or strokewidth
Due to ERROR: 
thread 'main' panicked at src/main.rs:40:10:
Compilation failed with error: TypstSource([SourceDiagnostic { severity: Error, span: Span(342140814152337), message: "current font does not support math", trace: [], hints: [] }])
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:35
  [2] execute(cmd::Cmd; inputcmd::Cmd, path::String)
    @ MakieTypstEngine ~/.julia/packages/MakieTypstEngine/z5zcV/src/rust_cli.jl:34
  [3] execute
    @ ~/.julia/packages/MakieTypstEngine/z5zcV/src/rust_cli.jl:20 [inlined]
  [4] compile_string(str::String, additional_font_paths::Vector{AbstractString})
    @ MakieTypstEngine ~/.julia/packages/MakieTypstEngine/z5zcV/src/rust_cli.jl:47
  [5] generate_typst_elements(input_text::Typstry.TypstStrings.TypstString, preamble::String)
    @ MakieTypstEngine ~/.julia/packages/MakieTypstEngine/z5zcV/src/render.jl:121
  [6] typstelems_and_glyph_collection
    @ ~/.julia/packages/MakieTypstEngine/z5zcV/src/render.jl:215 [inlined]
  [7] convert_text_string!(outputs::@NamedTuple{…}, input_text::Typstry.TypstStrings.TypstString, i::Int64, N::Int64, fontsize::Float32, font::FreeTypeAbstraction.FTFont, align::Tuple{…}, rotation::Quaternionf, justification::Symbol, lineheight::Float32, word_wrap_width::Float32, offset::Vec{…}, fonts::Attributes, color::ColorTypes.RGBA{…}, strokecolor::ColorTypes.RGBA{…}, strokewidth::Float32)
    @ MakieTypstEngine ~/.julia/packages/MakieTypstEngine/z5zcV/src/piracy.jl:26
  [8] (::Makie.var"#2906#2907")(::@NamedTuple{…}, changed::@NamedTuple{…}, cached::Nothing)
    @ Makie ~/.julia/packages/Makie/4JW9B/src/basic_recipes/text.jl:354
  [9] ComputePipeline.TypedEdge(edge::ComputePipeline.ComputeEdge{…}, f::Makie.var"#2906#2907", inputs::@NamedTuple{…})
    @ ComputePipeline ~/.julia/packages/ComputePipeline/03tW7/src/ComputePipeline.jl:126
 [10] #invokelatest#2
    @ ./essentials.jl:1055 [inlined]
 [11] invokelatest
    @ ./essentials.jl:1052 [inlined]
 [12] ComputePipeline.TypedEdge(edge::ComputePipeline.ComputeEdge{ComputePipeline.ComputeGraph})
    @ ComputePipeline ~/.julia/packages/ComputePipeline/03tW7/src/ComputePipeline.jl:120
 [13] (::ComputePipeline.var"#52#54"{ComputePipeline.ComputeEdge{ComputePipeline.ComputeGraph}})()
    @ ComputePipeline ~/.julia/packages/ComputePipeline/03tW7/src/ComputePipeline.jl:664
 [14] lock(f::ComputePipeline.var"#52#54"{ComputePipeline.ComputeEdge{ComputePipeline.ComputeGraph}}, l::ReentrantLock)
    @ Base ./lock.jl:232
 [15] resolve!(edge::ComputePipeline.ComputeEdge{ComputePipeline.ComputeGraph})
    @ ComputePipeline ~/.julia/packages/ComputePipeline/03tW7/src/ComputePipeline.jl:659
 [16] _resolve!(computed::ComputePipeline.Computed)
    @ ComputePipeline ~/.julia/packages/ComputePipeline/03tW7/src/ComputePipeline.jl:652
 [17] foreach
    @ ./abstractarray.jl:3187 [inlined]
 [18] (::ComputePipeline.var"#52#54"{ComputePipeline.ComputeEdge{ComputePipeline.ComputeGraph}})()
    @ ComputePipeline ~/.julia/packages/ComputePipeline/03tW7/src/ComputePipeline.jl:661
--- the above 5 lines are repeated 1 more time ---
 [24] lock(f::ComputePipeline.var"#52#54"{ComputePipeline.ComputeEdge{ComputePipeline.ComputeGraph}}, l::ReentrantLock)
    @ Base ./lock.jl:232
 [25] resolve!(edge::ComputePipeline.ComputeEdge{ComputePipeline.ComputeGraph})
    @ ComputePipeline ~/.julia/packages/ComputePipeline/03tW7/src/ComputePipeline.jl:659
 [26] _resolve!(computed::ComputePipeline.Computed)
    @ ComputePipeline ~/.julia/packages/ComputePipeline/03tW7/src/ComputePipeline.jl:652
 [27] resolve!(computed::ComputePipeline.Computed)
    @ ComputePipeline ~/.julia/packages/ComputePipeline/03tW7/src/ComputePipeline.jl:644
 [28] getindex
    @ ~/.julia/packages/ComputePipeline/03tW7/src/ComputePipeline.jl:563 [inlined]
 [29] #476
    @ ~/.julia/packages/Makie/4JW9B/src/compute-plots.jl:403 [inlined]
 [30] iterate
    @ ./generator.jl:48 [inlined]
 [31] _collect(c::Vector{…}, itr::Base.Generator{…}, ::Base.EltypeUnknown, isz::Base.HasShape{…})
    @ Base ./array.jl:811
 [32] collect_similar
    @ ./array.jl:720 [inlined]
 [33] map
    @ ./abstractarray.jl:3371 [inlined]
 [34] _register_expand_arguments!(::Type{…}, attr::ComputePipeline.ComputeGraph, inputs::Vector{…}, is_merged::Bool)
    @ Makie ~/.julia/packages/Makie/4JW9B/src/compute-plots.jl:403
 [35] _register_expand_arguments!
    @ ~/.julia/packages/Makie/4JW9B/src/compute-plots.jl:399 [inlined]
 [36] register_arguments!
    @ ~/.julia/packages/Makie/4JW9B/src/compute-plots.jl:377 [inlined]
 [37] (LineSegments)(user_args::Tuple{ComputePipeline.Computed}, user_attributes::Dict{Symbol, Any})
    @ Makie ~/.julia/packages/Makie/4JW9B/src/compute-plots.jl:742
 [38] _create_plot!(F::Function, attributes::Dict{Symbol, Any}, scene::Makie.Text{Tuple{…}}, args::ComputePipeline.Computed)
    @ Makie ~/.julia/packages/Makie/4JW9B/src/figureplotting.jl:410
 [39] linesegments!(::Makie.Text{…}, ::Vararg{…}; kw::@Kwargs{…})
    @ Makie ~/.julia/packages/Makie/4JW9B/src/recipes.jl:521
 [40] tex_linesegments!(plot::Makie.Text{Tuple{Vector{Point{3, Float32}}}})
    @ Makie ~/.julia/packages/Makie/4JW9B/src/basic_recipes/text.jl:452
 [41] calculated_attributes!
    @ ~/.julia/packages/Makie/4JW9B/src/basic_recipes/text.jl:431 [inlined]
 [42] connect_plot!(parent::Scene, plot::Makie.Text{Tuple{Vector{Point{3, Float32}}}})
    @ Makie ~/.julia/packages/Makie/4JW9B/src/compute-plots.jl:805
 [43] plot!(scene::Scene, plot::Makie.Text{Tuple{Vector{Point{3, Float32}}}})
    @ Makie ~/.julia/packages/Makie/4JW9B/src/interfaces.jl:211
 [44] _create_plot!(F::Function, attributes::Dict{Symbol, Any}, scene::Scene, args::Observable{Point{3, Float32}})
    @ Makie ~/.julia/packages/Makie/4JW9B/src/figureplotting.jl:411
 [45] #text!#64
    @ ~/.julia/packages/Makie/4JW9B/src/recipes.jl:521 [inlined]
 [46] text!
    @ ~/.julia/packages/Makie/4JW9B/src/recipes.jl:519 [inlined]
 [47] initialize_block!(l::Label)
    @ Makie ~/.julia/packages/Makie/4JW9B/src/makielayout/blocks/label.jl:10
 [48] _block(T::Type{…}, fig_or_scene::Figure, args::Vector{…}, kwdict::Dict{…}, bbox::Nothing; kwdict_complete::Bool)
    @ Makie ~/.julia/packages/Makie/4JW9B/src/makielayout/blocks.jl:405
 [49] _block
    @ ~/.julia/packages/Makie/4JW9B/src/makielayout/blocks.jl:321 [inlined]
 [50] #_block#1912
    @ ~/.julia/packages/Makie/4JW9B/src/makielayout/blocks.jl:266 [inlined]
 [51] _block(::Type{…}, ::GridPosition; kwargs::@Kwargs{…})
    @ Makie ~/.julia/packages/Makie/4JW9B/src/makielayout/blocks.jl:260
 [52] _block
    @ ~/.julia/packages/Makie/4JW9B/src/makielayout/blocks.jl:251 [inlined]
 [53] #_#1910
    @ ~/.julia/packages/Makie/4JW9B/src/makielayout/blocks.jl:242 [inlined]
 [54] Block
    @ ~/.julia/packages/Makie/4JW9B/src/makielayout/blocks.jl:241 [inlined]
 [55] #Label#2489
    @ ~/.julia/packages/Makie/4JW9B/src/makielayout/blocks/label.jl:1 [inlined]
Some type information was truncated. Use `show(err)` to see complete types.

Here’s my versioninfo:

versioninfo()
Julia Version 1.11.7
Commit f2b3dbda30a (2025-09-08 12:10 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 12 Ă— Intel(R) Core(TM) i7-10750H CPU @ 2.60GHz
  WORD_SIZE: 64
  LLVM: libLLVM-16.0.6 (ORCJIT, skylake)
Threads: 1 default, 0 interactive, 1 GC (on 12 virtual cores)
Environment:
  JULIA_EDITOR = code
  JULIA_VSCODE_REPL = 1
``
1 Like
5

This looks great! My main usecase for MakieTeX was actually to display Tikz images, so at least for my usecase this is more complementary than competitive :smiley:

But yeah, to avoid type piracy, an easy route could be that get_texelems_and_glyphcollection dispatches on some TextLayouter type as its first argument, which should be encoded in the attributes of the text plot. Then it’s very easy to switch the default layout engine either in the theme or in individual plots, without having type piracy - any layout engine can just define its own layouter struct.

6

I totally agree with you, having a library that we can pass strings in and get structs or dicts or arrays back from would be much cleaner and probably faster.

I looked a bit into creating a ABI layer, but at the time, I was more concerned about getting things to work at all. Additionally, as you can probably tell from the rust code, I do not really know how to write Rust. So far, it has been fun, but the current state feels very much like I duck-taped random things together until it did something.

In terms of speed, it is a fair bit faster than MakieTeX, and, considering that building a plot often means incrementally changing only a few of the strings to be rendered at a time, we might get the biggest speedup by just caching the results, and only pass stuff to the rust tool that changed.

7

The limitation for text/linesegments is in a way pretty artificial and only because MathTeXEngine just has lines, it would be conceivable to open this up into a plotlist kind of thing that can do polys and images, too, in which case one could cover a far larger share of typical situations for diagram drawing.

Hehe sure, but it’s fun to make things work in a quick and dirty way. Having a nice proof of concept gives you the motivation to dig into the details.

Makie is probably also sending some unnecessary updates that can be looked into, as long as the text only shifts in position and rotation, there should be nothing to recompute (as one can just move the same resolved layout around)

8

That is very interesting. For me (on MacOS) both fonts and font paths work and do what I want them to. The issue with TeXGyrePagella font is almost certainly an issue with the filepath that I can not see from here. In the end, it is calling into MathTeXEngine to reuse the files.

For the problem with LucioleMath, I think there might be a platform dependent issue in the way that typst discovers the fonts, compared to the way that makie does it. If you run the regular typst compiler with the correct font path:
typst compile --font-path /home/jonas/.julia/packages/MathTeXEngine/8VxpV/assets/fonts template.typ

on a file that looks like

#set page(margin: 1em, height: auto, width: auto, fill: white)
#set text(16pt, font: "JuliaMono")
#set text(font: "tex gyre pagella", 10pt)
#set align(center)

#show math.equation: eq => {
    set text(font: "TeX Gyre Pagella Math")
    eq
}

#show math.equation.where(block: true): eq => {
    set align(center)
    eq
}

this is an integral:
$ integral_0^t sin(x)^2 dif x $

does it still complain?

9

Thanks! I did not mean to position it as a competitor but as a more integrated alternative to put equations into makie that use the same font as my document :wink: While it was fiddly at times, I greatly appreciate that MakieTeX allowed me to do that at all.

I do think that the usecase of “putting arbitrary documents with graphics and pictures into the xaxislabel” is very very niche and have as such not yet tried anything more complicated than text and equations. For something like this, I think it still makes much more sense to just include an svg/pdf.

I think that makes sense. When I find some time over the weekend, I will see if I can draft a PR.

10

Using the typst executable generated in ~/.julia/artifacts/[...]/bin/typst I successfully compiled your example template.typ file using the command you suggested and got a .pdf that to my eyes (I’ve never used typst before) looks correct.

11

Assuming that [...] is the Typst_jll UUID, you can also call that binary from Julia with Typstry.typst("compile template.typ").

3 Likes
12

I think that with the new compute pipelines, you can check which arguments have changed? That might be a simple enough way to implement that, although there will be some points where things get marked dirty when they don’t need to.

The classic approach is to store the last seen value in a Ref and then just check equality but that can be a bit annoying to do sometimes, especially since types can change.

13

Is there some weird unicode-look-alike shenanigans going on here?

It’s much simpler than that. It’s just that the filenames are not properly capitalised, which causes problems on Linux. For instance, TexGyrePagellaMTE-Regular.otf should actually be TeXGyrePagellaMTE-Regular.otf (Tex → TeX).

This has been fixed in this PR, but the fix is still awaiting a new MathTeXEngine.jl release.

3 Likes
14

Thank you for spotting that. In spite of pasting the two strings right on top of each other in a text file, I somehow still didn’t see this difference.

15

:grimacing: I’m registering a new release as we speak, I forgot about it sorry.

Somehow, I think it would be better to have the whole layouting done before passing anything to Makie. But I can’t figure a sensible way to do it, while still allowing to choose the engine or font through theming :frowning:

On the other hand, I don’t thing that Makie currently allow to choose the MathTeXEngine font from plot attributes, so maybe it’s still the way to go.

1 Like