Call to write 1.9 extensions (and find pkgs that need those)

Community
13

This question is off-topic. We can discuss in private if you need further clarification. Bellow is a short answer:

These two packages are used in completely separate software stacks with different goals.

If you need advanced geometric processing besides visualization with Makie.jl then I strongly recommend using Meshes.jl.

MeshViz.jl provides the recipes to plot Meshes.jl geometries with tons of customization. These recipes call Makie.jl built-in functions, which in turn are implemented in terms of their internal GeometryBasics.jl types.

Similarly, MeshPlots.jl provides recipes which are implemented in terms of Plots.il built-in functions and geometries defined in the backends (e.g. GR).

It is really a matter of asking yourself what you want to do with these geometries. If you want to manipulate them in Julia and do further work before visualization, Meshes.jl is the most adequate choice.

1 Like
14

Gotcha. This distinction is pretty critical for other developers and fits within my previous comment pretty well. There needs to be text somewhere clearly stating this sort of stuff for the many packages that should have some form of plot support through an extension.

16

Understanding all of the pros and cons, and also the steps one needs to take to convert a package would be useful.

  • It would be useful to include the docstrings for functions in the extension in docs generated by Documenter.jl. I dont know how to do this. You can attach documentation to the declaration in the main package. Is this a perfect substitute? What about docs for calls with particular arguments?

  • What about linting and testing the extension pacakges, Test, Aqua, JET ?

  • If I understand correctly, you can’t cache native code for work that includes extensions. Is there a workaround? If not, you should be careful about prematurely converting a dep to a weak dep.

17

The first obvious approach is to look for packages that currently use Requires and replace that with extensions instead.

5 Likes
18

A first example could be

It uses Requires.jl to load optimization solvers with JuMP.jl, Convex.jl and Optim.jl.

2 Likes
19

Are there any cons? I don’t think there are any over using Requires beyond the time and effort to do the conversion (but, yes, I suppose the situation may be different if you’re converting a direct dependency to a weak one). Here’s a previous thread with a run-down on you you convert Requires to an Extension in a backwards compatible manner:

And as for that list, here are Requires’ dependencies (although this will also catch those that only use Requires for backwards compatibility):

Direct Dependents (566)

20

One con that I did notice is that with Requires one can actually define structs in the added code (now: extension) to the original package.
Sure that is not so reasonable, but took me a while to find a good alternative way to accomplish the same without introducing a new struct (that the parent package should even export).

But ± finishing that PR, one (more) of the 566 dependencies of Requires is soon based on extensions.

4 Likes
21

There’s a recent issue with Plots and IJulia where the extension doesn’t really work:

3 Likes
22

Out of curiosity, what was the alternative you found?

23

I remodelled the struct to be more general (in my case a struct that among others used some caches, now generic caches).
And I had to rethink/remodel a few functions that were working with said struct.

I am also not saying my original model was a good one (just because it was possible with Revise.jl), just that this rethinking and remodelling might be a con due to time constraints (eventually). So it might sometimes take a bit more time (especially when still learning details of extensions, which I definetly nearly know not much about).

1 Like
24

Seems like FileIO can use extensions for different file formats, isn’t it?

1 Like
25

Not exactly, it’s possible to keep a kind of backward compatibility, but this “requires” (pun intended) additional work, though.

5 Likes
26

If anyone can help on Loading main package from higher-level env fails with extension of package in current env · Issue #49886 · JuliaLang/julia · GitHub, it would be appreciated.

We moved to extensions in Plots in an effort to tidy up things, but the Ijulia problem might force us to go back to using Requires, and this is irritating.

1 Like
27

It would be nice to have a package that defines a macro to handle this backward compatibility. To be honest, the package extensions functionality feels a bit hacky for someone looking at it for the first time.

28

Could someone with the expertise take a look at MethodError in diff print: no method matching stat_rep(::Nothing) · Issue #3327 · JuliaLang/Pkg.jl · GitHub? This is leading to the invalidation check failing in two packages that I know of, which is stalling the work on package extensions. Presumably, other packages checking for invalidation using a standard script would also be impacted by this.

1 Like
29
5 Likes
30

@cjdoris how would you adapt the following PR to use the PackageExtensionTools.jl?

31

Only if you also want to provide the functionality to earlier julia versions (through Requires). This part is hacky indeed.
But package extensions by themselves look pretty clean, aside from a lot of unnecessary repetitions.
Like, really?? StructArraysStaticArraysExt = "StaticArrays" ref

32

I remember that there was a reason for the long name, that pkgs could clash if pkg A and pkg B both load a C extension as CExt. Using ACExt and BCExt seems to disambiguate names. But i don’t know if this is still a problem on the current release

33

Yeah, that was a PackageCompiler bug, didn’t affect normal usage of packages and is already fixed for some time (ref).
Another, remaining, case of repetition is the weakdep package name repetition (3x) in the common scenario of single-dependency extensions:

[weakdeps]
StaticArrays = ...

[extensions]
StaticArraysExt = "StaticArrays"