Seeking input (and creating output) about visualization software

Manifesto

I’ve been using Julia for about a year and a half now, and I truly love the language. More specifically, the REPL environment and the general customizability that the language provides are both amazing, especially the way that packages are then able interact with those two (ie. TwoBasedIndexing.jl or RemoteREPL.jl). Bundled with Julia’s incredible speed and beautiful syntax, these two features somehow overshadow the fact that the language only really has a decade of infrastructure behind it.

But one part of Julia, although well developed, seems to go against these very principles: GUIs. The packages attempting to fill this gap feel very static and slow. I love Makie.jl (and all the wonderful backends behind it) in a language designed around fast prototyping, it feels ridiculous to wait two minutes for a GLMakie plot to load. And CairoMakie.jl doesn’t fix this: although the TTFP is significantly faster than GL, it doesn’t create interactive windows, which rules out all sorts of CA or simulation visualizations.

This goes against what I consider to be the point of scientific visualization software: to be an easy yet versatile tool that will catch anything you throw and display it. We need a GUI library that will help you see what your code shows instead of forcing you to make the interface display what you think it should.

Design

These thoughts led me to design (but not yet code) my own GUI library. I’ll outline my current thoughts:

The core principle is that an instance of this GUI should be interactive, malleable software. This means not confining the user to the Figure system that Makie.jl uses or forcing the user to create a new plot for each visualization.
This idea is realized via a sort of infinite canvas.
To create a main canvas, one can simply use c = Canvas(). An empty window will appear.
Whenever the user defines a variable x that satisfies typeof(x) :> Drawable, the GUI will plot x on c (when a Drawable is created but not assigned to a variable, the GUI will open a new window to display that Drawable. This is to satisfy those who would like more customizability in their interface.)
Another important thing to note is that all Drawables are able to be moved around the canvas. Don’t like where you put something originally? Just click and drag.

This core idea of interactive-ness creates a WIDE range of possibilities. One could obviously zoom in and out on charts like in GLMakie, but why not also allow the user to modify the data in the GUI?

If this was built on top of Makie.jl, one could use observables in all sorts of ways: plot your simulation and then later add a slider to explore different time steps!

Conclusion

I can’t see any bad sides of this idea, probably because I have too much pride in it, (and because I’ve changed it every time I found one!) but I would love to hear your thoughts on it. How do you use GUIs? What parts of Makie do you like, and what parts do you find unnecessary? What about this package?

Regardless of the use of this idea, I do feel that visualization is an integral part of scientific research and we should think about how to make it easier. In the words of Mandlebrot:

Everybody in mathematics had given up for 100 years or 200 years the idea that you could from pictures, from looking at pictures, find new ideas.

This was written by a human. All complaints about existing visualization software are complaints, not attacks, no matter how aggravated they were. I just feel passionate about this.

What I like about Matlab Figures and would like to see in Julia.

  1. Matlab, so to speak, bundles both CairoMakie and GLMakie into one GUI. I can interact with the Figure, and then copy, via the clipboard, either the vector graphics or the bitmap graphics and paste it into a document. Yes I use Word as my document creation tool.
  2. Zoom and Pan are easy to use from its GUI.
  3. Datatips are hugely important, and being able to use them from the GUI interface is sooo convenient.
  4. Creating annotations from within the GUI is convenient.
  5. Missing from Matlab are harmonic and sideband cursors, which would be convenient for dealing with spectra. Delta cursors would also be convenient.

I sometimes wonder if it would be possible for Makie to send graphics to a window like the Matlab Figure window, which would be my ideal.

I have not used MatPlotLib so it may have some of this but I don’t know.

I have used the Raspberry PI. Hopefully the PI will at some point get a real graphics subsystem. Currently InspectDR is the most performant graphics library that I have used on the RPI 4 & 5. Unfortunately InspectDR still uses GTK 3 so is limited to Julia 1.10.

Hopefully this is helpful to your thinking.

Thank you! I love the Matlab Figure window (unfortunately I don’t have access to Matlab, but it seems great). My vision is close to that: a separate persistent window, but mine would diverge when we come to workflows.

I sometimes wonder if it would be possible for Makie to send graphics to a window like the Matlab Figure window, which would be my ideal.

If this was ever made, this is probably how it would be coded. A new Makie backend that has an infinite canvas and persistent, updatable windows.