Just a braindump from my blog, I’m interested in your thoughts on this.
In many programming languages, when you define a new type or object, you can also define how it behaves. But often, there’s no automatic way to check that your new type still behaves the way it’s supposed to — especially if it’s meant to “act like” another type.
In Julia, this is even more open-ended: you can subtype an abstract type and start overloading some other nerd’s functions, but nothing forces your implementation to meet expectations. This makes code flexible, but also fragile. You get a lot of freedom, but you’re roaming around in the Wild West.
Why don’t we use tests in Julia to define contracts? That means: when you define an abstract type (or any kind of interface), you also define a test suite that expresses the properties all subtypes should satisfy. Tests are assertions, mathematical propositions. Julia, with its powerful reflection capacities, could surely do something exciting with that.
For example, if something is a SortedCollection, it should always return its elements in order — and that should be part of its test contract. Then, when someone extends SortedCollection, those tests automatically run against their implementation.
This approach solves two problems at once. First, it prevents silent breakage: if a new type doesn’t behave as expected, it fails the contract. Second, it helps the compiler and runtime know what parts of the code are likely to be used — which helps with things like precompilation in Julia. In a language where dispatch is dynamic and types are open-ended, knowing the common execution paths ahead of time is probably a good thing.
I’d like some ready test suites if I implement an interface, but when and how do these tests run? I don’t want a test to run every time I instantiate or call a function, and it’s not possible to immediately run tests on a freshly defined type with no instances. It’s not possible to automatically instantiate edge cases, and there isn’t one general blueprint for designing them.
One aspect is to not run them as “tests”, but automatically on the arguments of certain high-level functions, unless check=false. This works in my case because optimize often runs for seconds, if not hours. You wouldn’t want to run interface checks every time inside performance-critical tight loops.
And that’s really the caveat with the approach: it works at runtime. If there was more Julia could do at compile time to verify formal interfaces, that would be a performance improvement.
The second aspect is that if as a user of QuantumControl I want to implement my own type that can serve as a “generator” of the dynamics (which the framework encourages), I can call check_generator to verify that everything is defined correctly to meet the expectations of the QuantumControl.propagate and QuantumControl.optimize functions.
I absolutely believe we should do more of that. If there’s some interface like, let’s say AbstractArray, there should be a pre-defined test suite that I can run my custom array-like type through to verify that it fulfills the interface.
There are other alternatives in the ecosystem, like RequiredInterfaces.jl, DuckDispatch.jl, probably others I’ve missed, and every package containing the word “trait”. A common standard has yet to emerge (cue XKCD meme).
.