I started working on “recipe” prototypes for MakieLayout but with a slightly different aim than what Plots offers. For me, a critical aspect is being able to do these things:
plot simple plots or complex plots with short commands
return everything that such commands create to the user to manipulate
– this includes the scene, top layout, plot layout, plot objects, maybe other elements like text etc. for complex recipes
–this to me is a weakness both of Plots and Makie at the moment, recipes can never completely satisfy an advanced user’s needs, so I want to be able to manipulate everything after its created
allow the user to reuse these elements easily to stack complex plots if they use compatible axis configurations
– example is plotting a 4x6 facet layout with one command, which returns the axis array and the gridlayout among other things so one can add a colorbar or legend into the sublayout and add more plots to single axes
Here’s an excerpt from a file where I was working on this with syntax examples:
# `infrastructure` could be one axis and a layout in the simplest case
# or many axes and support elements in arbitrary layouts in complex cases
# return types could be named tuples with a Base.display method overload
# so every of these return signatures could still cause display of the scene
scene, layout, infrastructure, plotobj = plot(args...)
layout, infrastructure, plotobj = plot(scene, args...)
infrastructure, plotobj = plot(layout[1, 1], args...) # layout has scene connected
plotobj = plot(infrastructure, args...)
There are challenges in this approach, too, but I liked the couple prototypes I did so far.
to define a way of plotting a user type (e.g. a regression model, phylogeny, neural network, shapefile map etc) (this is a user recipe), or
to define new plots (e.g. boxplots, candlesticks, dendrograms, streamplots, 2d histograms) etc. (this is a series recipe)
without enforcing a dependency (except RecipesBase which is just a few lines of code). If your recipe framework offers this, it is the same aim. If it doesn’t it’s probably not a recipe Everything else is extraneous or implementation details.
About “return everything that such commands create to the user to manipulate” I agree completely, but I think Plots does that to a large degree - recipes aren’t “closed” you can embellish them in any way you want in the call.
I think “plot complex plots with short commands” is also true for Plots (and should become so for Makie), and the same goes for “allow the user to reuse these elements easily to stack complex plots”.
Looking forward to talking about recipes at the vizcon!
Yes the basic ideas are very similar, but because of the way that Plots has to hand everything off to its backends, to me it feels quite “rigid”. It’s not so easy to manipulate elements after they’re created, to extend the layout (instead of re-plotting) and things like that. I’m not sure how different a recipe system has to look that wants to allow that. Will be interesting for sure to explore such questions together in Berlin!
I agree with that. One of the really nice things with Makie is the complete flexibility it gives. I’ve always thought the best part of Plots is not the backend shifting (which does give rigidity), it’s the recipes.
To my mind the ideal plotting package has
Syntax allowing fine-grained control of everything (like matplotlib - Makie is also in this category)
and at the same time very short terse syntax with excellent defaults, like Plots, for quick plotting
Complete freedom for visual styles, and
Multiple input paradigms
Greedy? I think it can be done, with recipes, Makie, MakieThemes and things like StatsMakie…
So there really seem to be two different design philosophies swirling around:
Option A:
Have Plots.jl be the user facing API
Write a backend for various plot packages
And then I guess Plots.jl handles the recipe stuff etc.
Option B:
Have different plot packages with different user facing APIs
Each of these plot packages can “consume” recipes
Makie.jl now seems to go into a direction that is more like B, right? Is that what we should do if we want be able to plot recipes with vega? Or is Option A better? I think it would be immensly helpful to me if someone could write down the pros and cons for these two approaches. And also, I’m entirely unclear whether option A is going away entirely? Or is that going to stick around? My understanding is that a major problem with A is speed, but then I’ve also seen reports that it has improved a lot.
I think option A is the existing reality for Plots. I think it’s going to stay around, given how succesful it is.
Option B is Simon’s vision for Makie. Given it’s a vision, it needs to be tested in reality (by actually implementing it). But if it works out I think it’s the ideal design, personally.
I think the strong suite for vega is it’s GoG syntax - I think that would be lost under option A.
We are planning to augment it with a whole bunch of “template” like functions that get you started more quickly, and some of them will look more like the Plots.jl API. We then have an idea for a GoG functional manipulation syntax for such a template that was created via a more imperative API. All in the early stages, but I have a pretty concrete idea how that will all look.
But I agree, Option B seems all around more attractive to me as well at this point.
So lets see whether @sdanisch’s strategy for Makie.jl can also be used for the vega stack!
Ok, one more question: How does the recipe stuff that @jules described fit into all of this? Is that yet another attempt that is independent from what @sdanisch is working on?
When you say different plot packages do you mean like Vega, Makie, Gadfly? If instead you mean each package has different user API within those ecosystems then people (me) are going to be very confused.
For now, we’re just talking about different plotting packages (I think). However, we will probably end up having a division of APIs (low-level v/s high-level), at least in Makie, and judging by what @davidanthoff is saying, in VegaLite as well.
What I mean is that Gadfly has an API, Makie has a (different) API, VegaLite has (yet another) API. In Option B all three packages would be capable of consuming recipes, though. I didn’t mean that any one package (say Gadfly) would be exposing different APIs.
I should add that my current thinkging about a QuickVega.jl package would essentially add another user facing API on top of VegaLite.jl, namely a non GoG API.
Dear all - the travel grants have been awarded, and it’s looking like it will become a great success - so far we’ve got 18 confirmed attendees, including many of the most productive and skilled contributors to Julia and it’s ecosystem. A list of attendees is found here Attendants for VizCon 2, Berlin 12-16 March · GitHub
If you’ve signed up, the main discussion channel is the #vizcon2 channel at the Julia slack. We’ll also send out an email with more information this Monday. Looking forward to it!
Dear everyone,
all universities of Berlin have decided to cancel all workshop meetings and conferences, as of immediately. That unfortunately affects the Vizcon, hosted at the Technical University of Berlin, who have cancelled the hosting of the event.
As you know, our policy throughout has been to follow the recommendations of the relevant authorities, in this case the WHO, the German government, and our host university. It is now beyond doubt that this requires us to cancel the meeting.
A number of us where already in Berlin. This includes myself, Lyndon, Glenn, Sam and Nicole. This is most unfortunate for this group, but also for the rest of us.
We hope we can still remotely do some work over the weekend on the plotting ecosystem, to enjoy the great impetus we’ve built up.
Just chiming in on recipes and their purpose. The purpose of recipes isn’t necessarily for writing a custom plot: one could create a new function for that. The purpose of recipes, and what’s unique about Plots.jl, is how they compose. I defined a recipe for ODESolution, Mose defined a recipe for Measurement, and there’s a recipe for Float32. Therefore, doing plot(sol) which is an ODE solution of Measurement{Float32} magically “just works”. In most plotting systems, someone would need to define each of the combinations, since the transformation of a Measurement to a value + error bars would be plot_measurement, and plot_odesolution wouldn’t necessarily know that it then needs to call plot_measurement (which itself would have to know to support Float32 and convert to Float64 for the eventual display!).
This is where the discussion really needs to be. Plots.jl isn’t necessarily the best plotting system, but this recipe idea is fantastic for a language that is based on multiple dispatch since all it’s doing is recursively hitting the same recipe function until the transformation gives it something that is a basic plotting primitive. Anyone then adds to the pipeline, and boom things compose.
To move beyond requires understanding this about the recipe system and bringing to other systems. Everything, from VegaLite to Makie, can be improved in its ability to handle arbitrary Julia code by support such a recipe system. The question is, what is the right way to do it? The one from Plots.jl has some weird issues (with layouts mostly), but the basic system in Makie doesn’t allow users to add new keyword arguments. This comes up for example in the Complex recipe in DimensionalPlotRecipes.jl gives users a way to change what kind of transformation you should do to a complex number: should you split it into two lines? Should you make a 3D plot? Should you do x vs norm(y)? The DiffEq plot recipe also introduces a bunch of keyword arguments, so the original idea of the Makie recipes was incompatible. However, the goal is for version 2 of Makie recipes to support enough of this, and I guarantee that DiffEq will support this new system.
Whether to support backwards compatibility with Plots.jl or not is a good question: there’s already a lot of Plots.jl recipes, so that would be helpful. But essential? That’s up to you all, but I will make sure I support whatever is found to be the right solution, and hopefully that means our DiffEq documentation can show ease of support for VegaLite, Makie, Plots, GR, etc. all in the near future, since our only tie to Plots is the recipe system (and without it, it would be fairly difficult to make plotting an ODE system work, given the weird types people throw at us).
You’re right in canceling the event. I’m so sorry this upended your careful preparations. To those traveling or in higher-risk areas, stay safe and know we’ll find ways to connect one way or another.
As most of you know, we’re holding a virtual mini-vizcon this weekend over zoom and github. Here’s a write-up of the first day:
We started with presentations:
Simon D introduced his ideas for a cleaner refactor of the AbstractPlotting conversion pipeline
Daniel gave an overview of recipes and how the pipeline works internally
Pietro described his work on StatsMakie combining recipes and Grammar of Graphics, launching an idea to make grammar of graphics interface completely independent of the plotting package.
After that a long good discussion where Josef suggested to reimplement recipes in a completely julia-independent way - as a yaml or json or css format. He would present some ideas about that tomorrow.
There’s a video of our first call here (Dropbox - File Deleted) . I didn’t edit the video at all (and sorry for filming myself from below), but talks start at minute 5.50.
We decided to work on StatsMakie, including the gog framework, and on trying to implement compatibility of Plots recipes in Makie.
The approach taken here was to move several functions from Plots (about half the pipeline.jl file) to a separate repo (mkborregaard/RecipeUtils.jl) that both Plots and MakieRecipes could depend on. That would make the entire recipe processing pipeline package independent - as long as a plotting package could implement 4-5 stub functions that dictate how to deal with defaults etc.
The StatsMakie group also managed to be close to having a working prototype of their implementation.
A lot of things are in flux, with a refactoring of AbstractPlotting and implementaiton with MakieLayout on the table.
Tomorrow we’ll have 4 short 15-minute talks in the morning, and then work on finalizing the projects. The talks will be
Josef: GR framework and new recipes
Julius: MakieLayout and philosophies
Simon C: lessons from the pgfplotsx backend to Plots
Juergen: VTK in julia
Let’s keep the talks short and sweet (trying to hit 15 minutes) to leave time for discussion and work tomorrow.
Thanks for a great day!
Everybody who wants can join tomorrow of course 10.am CET
I was glad to watch the recording on the first day. Especially, I found useful the general remarks and the discussion of the recipe. Did you happen to record the contributions of the second day?
Everything else is extraneous or implementation details.