1.9 Extensions are great, we need more of those. They are a great step into the dream of infinite interoperability that is one of my main motivators to use julia. For those reasons, i hereby make a call to extend as much packages as possible.
A first concrete step in that direction is a place to collect packages that could benefit of the use of extensions (if there is a GH issue, great!, make that issue visible here). This post is a good place as any to do that.
Putting this here as an easy way to ensure backwards compatibility with Requires: GitHub - cjdoris/PackageExtensionTools.jl: Makes Julia's package extensions backwards compatible
A related question: I wonder if it’s worth to turn very small and lightweight dependencies into extensions. Will it cause more overhead than any potential savings?
I’m talking about small packages like InverseFunctions or ConstructionBase.
No. Lightweight packages are fine. The place where you need extensions are where you want a small amount of code to make another large package work well.
However, more and more packages turn very lightweight dependencies into extensions: 1 2 etc. Should this be “officially” discouraged?
The ChainRulesCore (and similar packages like StaticArraysCore) are somewhat of an exception to this as they are packages that were made specifically to work around the fact that weakdeps didn’t exist. With a little luck, over the next couple years, all of those *Core packages will go away and we will just be using weakdeps on the full versions.
Ok, that was only one of my examples. What about InverseFunctions and similar, like ChangesOfVariables?
And even stdlibs sometimes get added as weakdeps.
Guess I should tidy and release it.
some package suggestions (just from my personal point of view):
This may actually be a good time to start putting together (whether for style guides or elsewhere) a list of recommended extensions packages based on what is being developed.
I didn’t have the time to read the package extensions functionality yet, but if anyone wants to help, we have a very concrete use case in our stack. We would like to move the Makie.jl recipes developed in MeshViz.jl to Meshes.jl so that whenever a Makie.jl backend is loaded in the same session, the recipes are loaded as well.
My main concern with package extensions so far is that the LTS version doesn’t support it. I understand that we need to drop support of LTS to move forward with the new language feature.
Is there a clear path forward with what’s going on between GeometryBasics.jl and Meshes.jl? As someone not involved in that development I find it confusing what I’m supposed to be doing with those packages.
This question is off-topic. We can discuss in private if you need further clarification. Bellow is a short answer:
These two packages are used in completely separate software stacks with different goals.
If you need advanced geometric processing besides visualization with Makie.jl then I strongly recommend using Meshes.jl.
MeshViz.jl provides the recipes to plot Meshes.jl geometries with tons of customization. These recipes call Makie.jl built-in functions, which in turn are implemented in terms of their internal GeometryBasics.jl types.
Similarly, MeshPlots.jl provides recipes which are implemented in terms of Plots.il built-in functions and geometries defined in the backends (e.g. GR).
It is really a matter of asking yourself what you want to do with these geometries. If you want to manipulate them in Julia and do further work before visualization, Meshes.jl is the most adequate choice.
Gotcha. This distinction is pretty critical for other developers and fits within my previous comment pretty well. There needs to be text somewhere clearly stating this sort of stuff for the many packages that should have some form of plot support through an extension.
Understanding all of the pros and cons, and also the steps one needs to take to convert a package would be useful.
It would be useful to include the docstrings for functions in the extension in docs generated by Documenter.jl. I dont know how to do this. You can attach documentation to the declaration in the main package. Is this a perfect substitute? What about docs for calls with particular arguments?
What about linting and testing the extension pacakges, Test, Aqua, JET ?
If I understand correctly, you can’t cache native code for work that includes extensions. Is there a workaround? If not, you should be careful about prematurely converting a dep to a weak dep.
The first obvious approach is to look for packages that currently use Requires and replace that with extensions instead.
A first example could be
It uses Requires.jl to load optimization solvers with JuMP.jl, Convex.jl and Optim.jl.
Are there any cons? I don’t think there are any over using Requires beyond the time and effort to do the conversion (but, yes, I suppose the situation may be different if you’re converting a direct dependency to a weak one). Here’s a previous thread with a run-down on you you convert Requires to an Extension in a backwards compatible manner:
And as for that list, here are Requires’ dependencies (although this will also catch those that only use Requires for backwards compatibility):
Direct Dependents (566)
One con that I did notice is that with Requires one can actually define structs in the added code (now: extension) to the original package.
Sure that is not so reasonable, but took me a while to find a good alternative way to accomplish the same without introducing a new struct (that the parent package should even export).
But ± finishing that PR, one (more) of the 566 dependencies of Requires is soon based on extensions.
There’s a recent issue with Plots and IJulia where the extension doesn’t really work: