I totally agree with you, having a library that we can pass strings in and get structs or dicts or arrays back from would be much cleaner and probably faster.
I looked a bit into creating a ABI layer, but at the time, I was more concerned about getting things to work at all. Additionally, as you can probably tell from the rust code, I do not really know how to write Rust. So far, it has been fun, but the current state feels very much like I duck-taped random things together until it did something.
In terms of speed, it is a fair bit faster than MakieTeX, and, considering that building a plot often means incrementally changing only a few of the strings to be rendered at a time, we might get the biggest speedup by just caching the results, and only pass stuff to the rust tool that changed.