We just had “Vizcon”, an open meeting for anyone interested in plotting and visualization with Julia. The idea came up at the Juliacon in London, and an open invitation was announced on the Slack #plotting channel, with follow-up planning at the Slack #vizcon channel. Present at the meeting were Simon Danisch (@SimonDanisch), Pietro Vertechi (@piever), Christof Stocker (@Evizero), Daniel Schwabeneder (@daschw), Moritz Schauer (@mschauer), Josef Heinen (@jheinen), Oliver Schulz (@oschulz), Sebastian Pfitzner (@pfitzseb), Diego Javier Zea (@diegozea) and myself (@mkborregaard). Most of us are associated with the JuliaPlots org, so we decided that the purpose of the meeting should be 1) discuss the future of plotting with Julia, 2) stabilize Plots for Julia 1.0, and 3) kick-start contributions to Makie.
Overall the meeting was a great success, with many important discussions and also quite a lot of actual work done. Here’s an overview of what happened.
One presentation and discussion involved the future of plotting in Julia. Plots is currently one of the most popular plotting packages in Julia, and as such very important for the viability of Julia as an all-in-one programming language solution for many users. In particular the underlying recipe system in the stdlib-like RecipesBase has been very successful, with just short of 400 reverse dependencies. It’s also quite difficult to maintain, because of the relationship to backend packages with very different interfaces. Over the last two years, we’ve reviewed merged on average 5 pull requests per week, and closed like 150 issues per year. But, issues are opened on Plots at roughly twice that rate, with no sign of diminishing returns, and there is currently >300 open issues on Plots. A presentation of this, and categorized tables of issues and pull requests, can be found here , here and here For long-term stability we thus need either more contributors, or, to find a less maintenance-heavy plotting solution.
At the Vizcon there was great enthusiasm about Makie as a solution for the future, which has a nice clear codebase and is really powerful in terms of interactivity and scope. It also has a system for recipes already and backends in the works. It is very possible that it can take the role of Plots in the future, but development is still open. Many of us present expressed an intent to spend some of our coding time contributing to Makie in the present and future, and that has already begun. Simon gave a guided tour of the code and design of Makie + AbstractPlotting.
Josef gave a presentation of GR, which is a much bigger framework than just a backend to Plots like many Julia users know it, acting as both backend and frontend for a major visualization pipeline. The presentation can be found here.
The biggest discussions we had were:
Precompilation and time-to-first-plot
The biggest weakness of Plots, and suffers from the issue that it’s hard to tell why this is so slow. Makie is currently even slower. The solution appears to be to get static package compilation to work. GR, which wraps a binary, is very fast from
using GR to first plot. We did some work to get Plots precompilation faster, i.e. #1825 and JuliaPlots/PlotUtils.jl#52 .
A new recipe system
The heart and soul of Julia is the primacy of custom types. We found that plotting in julia needs some plotting-package independent way of mapping custom types to a visualization in user packages, while only depending on a small stdlib. RecipesBase goes some of the way for that, but is very tightly linked to Plots and Plots syntax. We decided we need something that is (practically) dependency-free, uses the facilities of julia 1.0 (like named tuples), does not depend on macros, and which has an intuitive syntax that is close to how we tend to use plotting libraries. It should be general enough to be a stdlib usable by all julia packages - using the syntax would have to be done by the plotting package itself or some small glue package, not in a big metapackage like Plots does. We started working on the syntax, and Oliver now has an almost-working prototype: https://github.com/oschulz/plottingrecipes.jl/tree/dev and a prototype glue package for GR https://github.com/oschulz/PlottingGR.jl/tree/dev .
We also agreed that Makie needs a recipe system that is much more lightweight than the one currently offered by AbstractPlotting. This could be such a general recipe system but there are things to iron out, like Makie relies a lot on Observables at the very core but other plotting packages don’t. As a response to this, Simon implemented a change to the recipe system so the user will not have to worry about Observables: JuliaPlots/AbstractPlotting.jl#35 .
Declarative grammar / grouping
Many modern plotting packages like ggplot2 and vegalite have a declarative grammar, where you specify which plot elements should be control by which data, and then sensible plots are built automatically. An old idea is that both standard and declarative plotting syntaxes can be united (the library can be multi-paradigm) by expanding the
group ing behaviour of Plots. We talked about various ways this could be done, and Pietro now has a working prototype of this for Makie: JuliaPlots/StatsMakie.jl#7
Josef and Simon managed to developing a GR backend for Makie, can now be found here: https://github.com/JuliaPlots/GRMakie.jl .
A UnicodePlots backend is also planned.
Here is a list of pull requests made during the vizcon that have not been covered in the above:
Deprecate glvisualize backend
Transfer portfoliocomposition recipe
Implement frames axis argument
Allow recursive merging of plot Attributes
Add new example reference image
Initial version of GR Makie backend
New implementation of density, histogram, Table compatibility and lifting
Remove concept of atomics
Add linestyle for axes
Allow heatmap of nx1 or 1xn matrices
Fix docs build
Documenter compatibility update
Update reference images for testing
Allow customizable RenderMode for txt
Implement cycling color palette for series (/plots)
Testing UnicodePlots seriously
Allow to use arguments in default theme
• Apart from these things we also worked on various projects that are not covered by the PR’s above.
• Better Makie theming
• Work on getting the PlotlyJS Plots backend 1.0 compliant
• Refactor PlotRecipes to focus on graphs only (GraphRecipes) while moving other recipes elsewhere
• Remove the Plots/StatPlots dependency of PlotRecipes and get it 1.0 compliant.
Overall a very productive and fruitful meeting. The general sentiment afterwards was that meetings like this could be emulated, by ourselves or other organisations, to stimulate progress in the Julia ecosystem.
The NextJournal company in Berlin graciously allowed us to use their meeting and office facilities (and excellent coffee) for the meeting. We’re also very grateful to Julia Computing’s community funding for covering meals during the two meeting days.