# Tests as contracts?

**URL:** <https://discourse.julialang.org/t/tests-as-contracts/129350>\
**Category:** General Usage\
**Created:** [May 26, 2025, 2:35pm UTC](https://discourse.julialang.org/t/tests-as-contracts/129350 "2025-05-26T14:35:35Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![mvsoom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mvsoom/32/31051_2.png) [@mvsoom](https://discourse.julialang.org/u/mvsoom)\
**Post date:** [May 26, 2025, 2:35pm UTC](https://discourse.julialang.org/t/tests-as-contracts/129350/1 "2025-05-26T14:35:35Z")

</div>

> Just a braindump from [my blog](https://marnixvanso.om/tests-in-julia/), I’m interested in your thoughts on this.

In many programming languages, when you define a new type or object, you can also define how it behaves. But often, there’s no automatic way to check that your new type still behaves the way it’s supposed to — especially if it’s meant to “act like” another type.

In Julia, this is even more open-ended: you can subtype an abstract type and start overloading some other nerd’s functions, but nothing forces your implementation to meet expectations. This makes code flexible, but also fragile. You get a lot of freedom, but you’re roaming around in the Wild West.

Why don’t we use tests in Julia to define contracts? That means: when you define an abstract type (or any kind of interface), you also define a test suite that expresses the properties all subtypes should satisfy. Tests are assertions, mathematical propositions. Julia, with its powerful reflection capacities, could surely do something exciting with that.

For example, if something is a `SortedCollection`, it should always return its elements in order — and that should be part of its test contract. Then, when someone extends `SortedCollection`, those tests automatically run against their implementation.

This approach solves two problems at once. First, it prevents silent breakage: if a new type doesn’t behave as expected, it fails the contract. Second, it helps the compiler and runtime know what parts of the code are likely to be used — which helps with things like precompilation in Julia. In a language where dispatch is dynamic and types are open-ended, knowing the common execution paths ahead of time is probably a good thing.

---

<div class="post-metadata">

**Author:** ![Benny](https://avatars.discourse-cdn.com/v4/letter/b/49beb7/32.png) [@Benny](https://discourse.julialang.org/u/Benny)\
**Post date:** [May 26, 2025, 2:47pm UTC](https://discourse.julialang.org/t/tests-as-contracts/129350/2 "2025-05-26T14:47:04Z")

</div>

> [@mvsoom](#):
>
> Then, when someone extends `SortedCollection`, those tests automatically run against their implementation.

I’d like some ready test suites if I implement an interface, but when and how do these tests run? I don’t want a test to run every time I instantiate or call a function, and it’s not possible to immediately run tests on a freshly defined type with no instances. It’s not possible to automatically instantiate edge cases, and there isn’t one general blueprint for designing them.

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [May 26, 2025, 2:56pm UTC](https://discourse.julialang.org/t/tests-as-contracts/129350/3 "2025-05-26T14:56:04Z")

</div>

> [@mvsoom](#):
>
> Why don’t we use tests in Julia to define contracts? That means: when you define an abstract type (or any kind of interface), you also define a test suite that expresses the properties all subtypes should satisfy.

FYI, this is the approach chosen by Interfaces.jl.

---

<div class="post-metadata">

**Author:** ![goerz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/goerz/32/3269_2.png) [@goerz](https://discourse.julialang.org/u/goerz)\
**Post date:** [May 26, 2025, 3:00pm UTC](https://discourse.julialang.org/t/tests-as-contracts/129350/4 "2025-05-26T15:00:05Z")

</div>

I agree 100%, and I would encourage you to follow that approach.

> Why don’t we use tests in Julia to define contracts?

There’s nothing stopping you 😉.

I use this approach myself in the [QuantumControl packages via “interfaces”](https://juliaquantumcontrol.github.io/QuantumControl.jl/stable/api/quantum_control/#QuantumControlInterfacesAPI).

> [@Benny](#):
>
> when and how do these tests run?

One aspect is to not run them as “tests”, but [automatically on the arguments of certain high-level functions](https://github.com/JuliaQuantumControl/QuantumControl.jl/blob/faa8cb005268f028a6291aedd11542b871ca23b4/src/optimize.jl#L115-L134), unless `check=false`. This works in my case because `optimize` often runs for seconds, if not hours. You wouldn’t want to run interface checks every time inside performance-critical tight loops.

And that’s really the caveat with the approach: it works at runtime. If there was more Julia could do at compile time to verify formal interfaces, that would be a performance improvement.

The second aspect is that if as a user of `QuantumControl` I want to implement my own type that can serve as a “generator” of the dynamics (which the framework encourages), I can call [`check_generator`](https://juliaquantumcontrol.github.io/QuantumControl.jl/stable/api/quantum_propagators/#QuantumPropagators.Interfaces.check_generator) to verify that everything is defined correctly to meet the expectations of the `QuantumControl.propagate` and `QuantumControl.optimize` functions.

I absolutely believe we should do more of that. If there’s some interface like, let’s say `AbstractArray`, there should be a pre-defined test suite that I can run my custom array-like type through to verify that it fulfills the interface.

---

<div class="post-metadata">

**Author:** ![mvsoom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mvsoom/32/31051_2.png) [@mvsoom](https://discourse.julialang.org/u/mvsoom)\
**Post date:** [May 26, 2025, 4:29pm UTC](https://discourse.julialang.org/t/tests-as-contracts/129350/5 "2025-05-26T16:29:13Z")

</div>

Great! I searched around a lot for something like this, thanks for pointing me to it.

---

<div class="post-metadata">

**Author:** ![ffevotte](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ffevotte/32/6587_2.png) [@ffevotte](https://discourse.julialang.org/u/ffevotte)\
**Post date:** [May 26, 2025, 4:30pm UTC](https://discourse.julialang.org/t/tests-as-contracts/129350/6 "2025-05-26T16:30:44Z")

</div>

Here is a related Invenia blog post:

> **[Development with Interface Packages](https://invenia.github.io/blog/2020/11/06/interfacetesting/)**
>
> Over the last two years, our Julia codebase has grown in size and complexity, and is now the centerpiece of both our operations and research. This implies that we need to routinely replace parts of the system like puzzle pieces, and carefully test if...

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [May 26, 2025, 5:47pm UTC](https://discourse.julialang.org/t/tests-as-contracts/129350/7 "2025-05-26T17:47:22Z")

</div>

There are other alternatives in the ecosystem, like RequiredInterfaces.jl, DuckDispatch.jl, probably others I’ve missed, and every package containing the word “trait”. A common standard has yet to emerge (cue XKCD meme).