Defining extensions

General Usage
,
1

I have a package extension that defines a StructTypes.jl interface for a type I do not own so that I can use JSON3.jl on it. What if another package does the same and both are loaded?
What would be the correct way of dealing with this?

module Foo
import SomePackage
struct MyFoo
  a::SomePackage.SomeType
end
end

module FooJSON3Ext
import SomePackage
import JSON3.StructTypes
StructTypes.construct(::Type{SomePackage.SomeType}) = ...
end
2

This is type piracy. You should not do this.

What if another package does the same and both are loaded?

There be dragons :dragon: :pirate_flag: :fire:

What would be the correct way of dealing with this?

The extension should live either in StruuctTypes.jl, or in JSON3.jl

2 Likes
3

It’s vague but I think the implication is that SomeType is in an unnamed package, and it’s not documented by either StructTypes or JSON3. module Foo doesn’t define or import it either, so I’m not sure.

If you try to add them to the same environment, then you run into a Method overwriting is not permitted error during precompilation. Julia deliberately does not want loading order to determine which duplicated precompiled method survives in a session; not only is loading order much harder to control than people would think, the precompile cache wastes space storing N-1 precompiled methods that are never used per session. The error suggests __precompile__(false) to opt the package out of precompilation so it doesn’t bloat the cache, but it consequently makes the package unable to be imported and built upon by other precompiled packages. You usually don’t want to sacrifice that feature of your package, whether for yourself or other people.

If you try to dodge this by tweaking the method signatures to be different somehow, you’re still doing type piracy, which risks breaking method dispatch assumptions that other packages need to work. And you still can’t anticipate someone else making yet another method that matches one; avoiding type piracy is just good manners for sharing code in packages.

4

It would be nice if JSON3.jl could simply serialize anything, but it cannot.
Shouldn’t it be actually an extension for the foreign type in its package? So neither in JSON3.jl nor StructTypes.jl?

5

I updated my original post, sry.

6

So I see three solutions

  • Try to not use the foreign type
  • Define the StructTypes.jl interface on the type I own, and skip the nested field serialization.
  • Open a MR in SomePackage to either make the type serializable or add an extension which defines it
7

I think this is the right approach and in the meantime you can put that extension code in your package as well to be able to continue your work without too much hassle. Yes this is type piracy, but you are aware of it and it only a temporary bandaid. The moment your PR is merged, you remove the type piracy from your package and everything is fine (and if the compat bounds are configured correctly even every valid combination of the two packages is fine).

I think “don’t do type piracy” is a bit too dogmatic. It should at least have an asteric saying “unless there is currently no other way and you are aware of the consequences” or so.