Best practice for user-customizable methods

General Usage
,
1

Suppose there is a function in some package for which I want to implement a custom method in my own package MyPackage. However, I also want users of MyPackage to customize/overwrite that method further if they want. I’m wondering what is the best practice for allowing this.

As a specific example, let’s take DrWatson’s default_prefix function. As discussed in the documentation of that function, let’s say I want to customize it for a MyType in MyPackage.

Option 1

One possibility is to define the following in MyPackage:

import DrWatson
default_prefix_mytype(c) = "lala"  # no type annotation!
DrWatson.default_prefix(c::MyType) = default_prefix_mytype(c)

I can then tell users of my package that if they want to overwrite the default_prefix for MyType, they should define their own

import MyPackage
MyPackage.default_prefix_mytype(c::MyPackage.MyType) =  "user_lala"

Since that’s more specific than the default_prefix_mytype without any type annotation, it will essentially overwrite the method in MyPackage.

Option 2

The other possible solution would be to directly set

import DrWatson
DrWatson.default_prefix(c::MyType) = "lala"

in MyPackage and to tell my users that if they don’t like that, they should just define their own

import DrWatson
using MyPackage: MyType
DrWatson.default_prefix(c::MyType) = "user_lala"

This would exactly overwrite the existing method.

The first method seems cleaner, but more convoluted to communicate in the documentation. The second method is easier to communicate, but it raises the question of how exactly Julia handles overwriting methods. Which method definition wins? I assume the last using wins, right? If a user does using MyPackage in their script, then overwrites the method, and then calls using MyPackage again (directly or indirectly), the second using is ignored, right? (Or does it revert the method definition?).

What is the recommended best practice in a situation like that? Is there some more elegant third option?

2

Both of these are type piracy which is recommended against because it makes the behavior of code harder to understand. In both cases, the behavior of default_prefix(::MyType) changes based on whether the user’s code was loaded, even though neither default_prefix nor MyType comes from the user code.

Given that they both have the same type piracy issue, I can’t see any reason to pick Option 1–it has the same drawbacks just with a whole extra layer of complexity that you have to explain to your users.

The second method is easier to communicate, but it raises the question of how exactly Julia handles overwriting methods. Which method definition wins? I assume the last using wins, right?

Actually both options raise this question, since a user might have two pieces of code (perhaps from yet another package which uses MyPackage) that both try to overwrite default_prefix_mytype. And yes, the last method definition wins, which is why both designs are inherently fragile.

So if this is really the behavior you want, where a user is encouraged to change the behavior of MyPackage (not adding new behaviors for new types, but changing the behavior of existing types), then Option 2 is better than Option 1. But it’s worth considering whether this is actually what you want, since this is essentially a way of mutating global state, which is an endless source of confusion.

Is there another way to express the kind of thing you actually want to achieve?

3 Likes
3

What I have in mind is not a package author building on top of MyPackage, but an “end-user”. In that context, I feel like type piracy may be a bit more appropriate.

What I actually want to achieve is in fact specifically customizing (and letting the “user” customize) how the DrWatson package chooses filenames: In my own package QuantumControl (or QuantumControlBase) – which is the real MyPackage – I have an OptimizationConfig struct. This struct encodes an optimization problem, and the result of that optimization is written to a file by DrWatson’s writing tools like @produce_or_load, where the automatic filename is determined by the methods DrWatson.default_prefix(c::OptimizationConfig), DrWatson.allaccess(c::OptimizationConfig), DrWatson.allignore(c::OptimizationConfig), DrWatson.access(c::OptimizationConfig, key) and DrWatson.default_allowed(c::OptimizationConfig).

Within QuantumControlBase, I try to provide these methods to result in reasonable default filenames. Right now, these are done with “Option 1”.

The “user” is someone using the QuantumControl package in a DrWatson-based project, that is, basically a collection of scripts doing various optimizations, and storing the result to disk. I think it would be at least somewhat common for that user to want to customize the automatic filenames for the stored results, for that particular project. I don’t see any reasonable way for them to do that without some type piracy.

It seemed like the more “blatant” piracy of “Option 2” was maybe going to behave a bit more unpredictably. The potentially tricky situation would be a script file in the project that starts like this:

using DrWatson
using ProjectSrc   # uses QuantumControl/QuantumControlBase, overwrite methods
using QuantumControl # uses QuantumControlBase

The ProjectSrc here is their project specific code (the src folder of the DrWatson project), which includes custom methods overriding the originals in QuantumControlBase. Now, would the third line reset the overwritten method from line 2? With “Option 1”, definitely not, but with “Option 2” I’m actually not sure what happens.

Maybe I’m just worrying too much, and in practice users will manage just fine with “Option 2” (If lines 2 and 3 in the above example are switched and using ProjectSrc is last, there is never a problem). I’m definitely not worried about a bunch of third-party packages implementing conflicting methods for OptimizationConfig, which is a situation where type piracy would completely blow up.

4

Do you need the user to have complete control of the function? One way that packages like Plots and Latexify deal with customization is to have a global (gasp) Dict of defaults, which you edit with a user-facing setter.

For your usecase, that could be

export set_mytype_prefix
global mytype_prefix = Ref{String}("lala")
function set_mytype_prefix(new)
    mytype_prefix[] = new
end
Watson.default_prefix(c::MyType) = mytype_prefix[]
2 Likes