Interfaces/traits in julia 2.0 and multiple inheritance

Recently, I started programming in Rust, and like many others, I noticed how powerful the design language and solutions around traits can be. This prompted me to investigate Julia’s interface and trait-related packages:

  1. CanonicalTraits.jl
  2. SimpleTraits.jl
  3. BinaryTraits.jl
  4. Interfaces.jl
  5. DuckDispatch.jl
  6. WhereTraits.jl

and questioning whether Julia even needs the nominal type system While “nominal type system” might not be the best term for Julia’s type hierarchy, I’m referring here to its explicit single-inheritance abstract type design.

Building a Rust-like but more flexible Type System in Julia

It turns out that it’s possible to construct a Rust-like type system in Julia, though there are notable limitations. For instance, DuckDispatch.jl doesn’t support multiple trait inheritance, which feels unnatural for a trait system. This limitation is understandable, as Julia itself lacks multiple inheritance, making it challenging to express such concepts.

On the other hand, WhereTraits.jl allows dispatch based on arbitrary conditions, but you need to manually define the trait hierarchy. So far, WhereTraits.jl seems to be the closest candidate for implementing a practical, Rust-like trait-based system. Here’s an example of how to construct a trait-based (Rust-like) type hierarchy using WhereTraits.jl:

Example
using WhereTraits

ATrait(::Type{T}) where T = isdefined(Main, :func1) && isdefined(Main, :func2) && hasmethod(func1, Tuple{T,Int64}) && hasmethod(func2, Tuple{T,Int64,Int64})
BTrait(::Type{T}) where T = isdefined(Main, :func3) && hasmethod(func3, Tuple{T,Int64,Int64,Int64}) && ATrait(T)
CTrait(::Type{TA}, ::Type{TB}) where {TA, TB} = ATrait(TA) && BTrait(TB) && hasmethod(func_ab, Tuple{TA, TB})

@traits_order (Main).process(x::T) where T begin
    BTrait(T)
    ATrait(T)
end


@traits function process(x::T) where {T, !ATrait(T)}
    42
end

@traits function p(x::T) where {T, ATrait(T)}
    100
end

@traits function process(x::T) where {T, ATrait(T)}
    func1(x, 1) + func2(x, 2, 3) + p(x)
end

@traits function process(x::T) where {T, BTrait(T)}
    func1(x, 1) + func2(x, 2, 3) + func3(x, 4, 5, 6) + p(x)
end

@traits func_ab(x::TA, y::TB) where {TA, TB, CTrait(TA, TB)} = 1 

@traits function process(x::TA, y::TB) where {TA, TB, CTrait(TA, TB)}
    func_ab(x, y) + process(x) + process(y)
end


struct ConcreteB end
func1(x::ConcreteB, y::Int64) = y
func2(x::ConcreteB, y::Int64, z::Int64) =  3 + y + z
func3(x::ConcreteB, y::Int64, z::Int64, w::Int64) = 6 - y - z - w


struct ConcreteA end
func1(x::ConcreteA, y::Int) = 1 + y
func2(x::ConcreteA, y::Int, z::Int) = 3 + y + z

struct Concrete end

# Test the dispatch
x1 = Concrete()
x2 = ConcreteA()
x3 = ConcreteB()
println(process(x1))
println(process(x2))
println(process(x3))
println(process(x2, x3))
println(process(x3, x2))

Challenges with Julia’s Current Type System

Julia’s reliance on single-inheritance hierarchies often forces users to cram unrelated concepts into a single hierarchy, which can be infeasible and lead to convoluted designs. For example, consider this notorious “dispatch monster” for matrix multiplication:

Dispatch monster
*(X::Union{ReshapedArray{TX,2,A,MI} where
MI<:Tuple{Vararg{SignedMultInverse{Int64},N} where N} where
A<:DenseArray, DenseArray{TX,2}, SubArray{TX,2,A,I,L} where
L} where I<:Tuple{Vararg{Union{AbstCartesianIndex, Int64,
Range{Int64}},N} where N} where
A<:Union{ReshapedArray{T,N,A,MI} where
MI<:Tuple{Vararg{SignedMultInverse{Int64},N} where N} where
A<:DenseArray where N where T, DenseArray},
 A::SparseMatrixCSC{TvA,TiA}) where {TX, TvA, TiA}=

This complexity highlights the ergonomic challenges that arise from Julia’s lack of support for orthogonal trait hierarchies and multiple inheritance. The absence of multiple inheritance forces its users to put concrete type into a single hierarchy even when it is not feasible. While I understand that adding multiple inheritance would require a significant overhaul of Julia’s type inference engine and related systems, I still feel that its omission was a missed opportunity. What’s the big deal in having yet another source of ambiguity if MethodError: method is ambiguous. already exists.

Questions

  • What are the chances that Julia 2.0 will move away from its brittle single-inheritance hierarchy and adopt a more robust, trait-based type system? This already seems feasible in Julia 1.x, aside from ad hoc handling of multiple inheritance.
  • Where can I find discussions related to the design and goals of Julia 2.0?
  • Is Julia 2.0 even being discussed seriously at this point?
  • What is the best way to contribute to implementing features like a trait-based type system in Julia? How can I effectively begin learning the details of language implementation? Simply poking around random parts of the GitHub repository feels somewhat inefficient.
6 Likes

No. Any discussion of v2 has the tacit acknowledgement that it can’t be done in v1 and isn’t being planned at all. Nothing stops you from working on it, it just means the other developers are prioritizing new ground in v1, so far with no end in sight.

Definitely find another term because nominal typing is not nearly as controversial as you’re making it out to be. Rust’s traits are nominal, not structural or something else, and they don’t obviously improve the Union dispatch monster example predating v1. You can probably find a current example by reading the source or trying various methods calls, just know the same complexity may appear less so because the where printing got more concise and implicit in v1.

3 Likes

Is Julia 2.0 even being discussed seriously at this point?

No, not as far as I am aware. It’s mentioned as a hypothetical thing, but I believe none of the core devs are considering it at the moment. There is still so much stuff to do in the 1.x series, it would make no sense to make a breaking change at this point.

Where can I find discussions related to the design and goals of Julia 2.0?

There is no such official document, because it’s not being seriously planned. I believe Stefan Karpinski have said he had a document with his own ideas somewhere. However, that’s not an official design document.

Broadly speaking, I agree that Julia’s inheritance model is not particularly good. Most of the interesting properties of types are not monophyletic.
But I do believe you’re underestimating the problem of ambiguity that would result from crossing Julian multiple dispatch with a trait system. I fear method ambiguities would explode.

Another thing you might consider is that duck typing also solves a large part of the problem in practise.
Consider something like Iterators.flatten. In Rust, the signature would be something like (excuse my non-expert Rust-ness)

fn flatten<T, E, S>(it: T) -> impl Iterator<Item=E>
where
    T: Iterator<Item=S>,
    S: Iterator<Item=E>,

This cannot be expressed in Julia because we don’t have an iterator trait. But here is the banger: We don’t need to express it in Julia. In Julia, the signature can just be:

function flatten(it)

Because we can duck-type.
Yes, that has some issues:

  1. It’s less self-documenting what kind of data it takes
  2. It makes it much harder to do static analysis

But one thing that’s NOT an issue is that it doesn’t really pose a problem for specialization and multi-methods - at least, not usually. The generic method above that takes any iterable of iterable doesn’t need a type signature precisely because it’s the default fallback, which only uses the behaviour of it.
If we want to write specialized methods of flatten, it would be for specific types in order to leverage their peculiarities, probably not for abstract types.

8 Likes

For what it’s worth, we had this conversation already rather extensively in Why did Julia choose nominal typing over structural typing/traits? and the response was more or less that the OP did not believe that this would actually be a problem. :person_shrugging:

2 Likes

One could encode peculiarities into a trait though, however I can see how it can be awkward sometimes. Nevertheless, it is a common thing even now for libraries to expose an abstract type that has only one concrete type implementation just to make the library more extensible just in case someone want’s to use it even with more peculiar concrete type.

Yes, I am probably missing something obvious here, as I don’t quite understand why this would lead to an exponential explosion of ambiguities. I thought you could always resolve an ambiguity by creating a trait that inherits from multiple other traits, effectively specializing the ambiguous function.
For example, Rust is planning to implement the trait specialization feature, which effectively provides static multiple dispatch. I presume they are not concerned that this is an issue either.
BTW, @Mason, thank you for contributing to the conversation, even if it might feel to you like you’re repeating the same obvious points. I assure you, my curiosity is genuine - I’m truly interested in understanding how the language can be improved.

1 Like

Assuming that we always could for a moment, it’s usually not a good approach to introduce another trait, supertype, method, etc. just for disambiguating. That’s how you get bloated APIs and poor code reuse. Disambiguation in other languages is sparingly used when refactoring isn’t feasible or economical. Even in Julia, defining a more specific method just to resolve a method ambiguity is a sign the dispatch design is going the wrong way, definitely if all you intend to do is invoke one of the ambiguous methods.

2 Likes

Consider three different possibilities:

  1. We have traits Iterable and AbstractArray. The latter inherits from Iterable, and so is a subtype. In that case, there is no ambiguity between f(::AbstractArray) and f(::Iterable). No problem!

  2. We have disjoint traits: HasFastIn (for e.g. sets and ranges), and Ordered. Then we could have two distinct implementations: isdisjoint(::HasFastIn, x, y) and isdisjoint(::Ordered, x, y), using two distinct algortihms. Which one should be used for types that are both, like ranges?

  3. What then, if we are able to dispatch on both abstract types AND traits? How can we decide if AbstractSet is more or less specific than e.g. a BitsType trait?

2 Likes

What you describing here is when traits intersect on the concrete types but each of them is not a subset of others:

 +-----------------------------------------+
|[impl<T:Copy> Clone for T]               |
|                                         |
| Example: i32                            |
| +=======================================+-----+
| |[impl<T:Copy> Clone for Option<T>].....|     |
| |.......................................|     |
| |.Example: Option<i32>..................|     |
| |.......................................|     |
+-+=======================================+     |
  |                                             |
  |   Example: Option<Box<i32>>                 |
  |                                             |
  |          [impl<T:Clone> Clone for Option<T>]|
  +---------------------------------------------+

It seems that this ambiguity screams for some efficient specialization if both properties exist. This is truly an ambiguous case. But for anything else, there exists a logical and reasonable algorithm to figure out which method to use. E.g. Intersection Impls
Distinguishing reuse from override

  1. What then, if we are able to dispatch on both abstract types AND traits? How can we decide if AbstractSet is more or less specific than e.g. a BitsType trait?

Yes, I am not sure how would it work if the language exposes both traits and abstract types, I think having just traits is enough to achieve the same functionality as with abstract types.

Yes, you can always write more specific signatures, but as the number of overlapping methods increases, the number of specialized methods you need to write to resolve the ambiguity increases exponentially (or combinatorially? I forget)

That’s bad enough, but the problem is much worse than just the fact that you need to write more methods to resolve ambiguities. There’s also the problem of who is supposed to write those methods.

If Base defines a function f, and some traits T1 and T2, e.g.

Base.f(::Base.T1, ::Base.T2) = ...

and then package B comes along and writes some additional trait methods on f, with their e.g.

Base.f(::B.T3, ::B.T4) =...

then they might create some ambiguities, and thus need to write some specializations like

T13 = SomeIntersectionOf(Base.T1, B.T3)
T24 = SomeIntersectionOf(Base.T2, B.T4)
Base.f(::T13, ::T24) = ...

Now suppose package C also exists and was doing the same sort of thing where it defined some traits on f using C.T5 and C.T6. It might also need to write some additional methods to patch ambiguities between it and Base, but it also might need to account for the fact that package B might be loaded and suddenly create new ambiguities.

E.g. we’d potentially also need not just

T15 = SomeIntersectionOf(Base.T1, C.T5)
T26 = SomeIntersectionOf(Base.T2, C.T6)
Base.f(::T15, ::T26) = ...

but also

T35 = SomeIntersectionOf(B.T3, C.T5)
T46 = SomeIntersectionOf(B.T4, C.T6)
Base.f(::T35, ::T46) = ...

(and of course, there’s many ways two traits can intersect, so the number of intersection types can also increase).

So B and C are independent packages that only know about Base, but if a user loads them both, then it’s possible (and likely!) that they will start interfering with each-other and create new ambiguities.

So there’s not only a potentially large increase in the number of possible ambiguities, but also the way those ambiguities appear can be non-obvious to the people actually writing the methods, since not every package knows about every other package.

3 Likes

I’ll point out that the area where we currently experience the most ambiguous methods right now – linear algebra, absolutely is plagued with the problem of needing real specializations and having real ambiguities (in addition to the ones that could hypothetically be solved algorithmically with some help from annotations), and this is just with single inheritance. It could potentially be much much worse with multiple inheritance.

The other thing I’ll point out is that Rust is a rather poor guide for Julia here, because (as far as I understand), Rust’s trait system is quite limited in some ways that makes a lot of these problems much less prevelant, but those limitations would be highly unappealing to Julia users.

1 Like

But isn’t this already happening:

 f(x::Int, y::Any) = 42
 f(x::Any, y::Int) = 69
 f(42, 69)
ERROR: MethodError: f(::Int64, ::Int64) is ambiguous.

Candidates:
  f(x, y::Int64)
    @ Main REPL[31]:1
  f(x::Int64, y)
    @ Main REPL[29]:1

Possible fix, define
  f(::Int64, ::Int64)

I agree that the number of dimensions where such ambiguities can occur increases from 1 (current abstract type hierarchies) to some number N with traits. However, if the Base trait is carefully designed, it shouldn’t be much of an issue.

I see, so your point is that it’s already an issue with one source of ambiguity, let alone when there are multiple. Yet, it feels like it shouldn’t be an issue.

Yes it’s already happening, and it’s already bad (as I’ve said many times about the linear algebra ecosystem). Adding multiple inheritance creates more ways for methods to ambiguously intersect.

However, the sort of traits you’re describing here, where a type would automatically become a member of a trait post-hoc if it satisfies the traits interface would have the additional problem that by merely loading a package, signatures that were once concrete can suddenly become ambiguous, because suddenly a concrete type could end up having more traits attached to it than it used to have.

I think part of the problem here is that the issue I’m describing only appears at scale.

I guess my hope was that giving toy examples with a small number of traits, types, and methods would help you imagine how this problem would scale as the number of traits and the number of concrete types and the number of methods on a function increases, but it appears to not be working.

I simply don’t have time or the will to create a realistic example with an ecosystem of 20 different concrete types and 10 different overlapping traits spread across 5 different packages just to demonstrate that the surface area of overlapping ambiguities could become cripplingly large.

3 Likes

Maybe it’d help if I shared this graphic:

using LazyArrays, ApproxFun, SparseArrays, LinearAlgebra, StaticArrays, StructArrays
using Plots, GraphRecipes
plot(AbstractArray, method=:tree, fontsize=3, nodeshape=:rect)

I really shudder to imagine how we’d handle this beast with multiple inheritance.

8 Likes

Now that we can invoke specific MethodInstances on master, couldn’t that be a solution? Sure, there would still be ambiguities, but breaking them would be “as simple” as defining a method that breaks the ambiguity with a decision on how each type should behave for that particular combination. If there is a most generic fallback that would be applicable [1], we could even invoke that by default. If there is no such method, you end up with a regular ambiguity.


[1] To make this clearer: an ambiguity is reported when, during dispatch, more than one method candidate is found that the call signature could dispatch to. In this idea of “invoke the most generic one”, you’d invoke say *(::AbstractArray, ::AbstractArray) instead of a specialized version. This will lead to non-optimal code, but could help prevent “ambiguity explosions”.

invokeing a MethodInstance is more of a code-generation strategy rather than a dispatch tool (MethodInstances are something generated relatively late in the compiler. Generating and invokeing them is a way to create something like generated functions but after type inference). You’ve been able to invoke specific methods for ages with regular invoke.

Even if you just end up writing invoke though, you still have to actually write a specific method that then calls invoke, which means writing lots of extra methods. E.g. going back to this simple example:

you’d still need to write the method f(x::Int, y::Int) = invoke(f, Tuple{Int, Any}, x, y). A problem with multiple inheritance is how it’d increase the number of such methods you’d potentially need to write.

That’s definitely an option, but remember that if we have multiple inheritance, then finding a “most generic fallback” could be just as hard as finding a “most specific method”, so I suspect that option could also be quite ambiguous.

The other problem is that people usually write specializations for a reason (they want those specializations to be hit!) so having your specializations silently get skipped because of an ambiguity sounds just as frustrating as hitting an ambiguity error (though frustrating in a different way).

1 Like

Do you think implementing rules similar to Rust’s orphan rules (which prevent type/trait piracy) could help alleviate this issue? Essentially, it would ensure that no ambiguities are created simply by loading packages B and C together. Any ambiguities would only arise from the user’s actions, which they would be responsible for resolving.

1 Like

If the ambiguous parts of the signature only have Any as their least common ancestor as in your Int/Any examples, then yeah, it’ll always be ambiguous and you will have to break the ambiguity manually. But if they do share a common one, as is the case in all of the types in LinearAlgebra, it should work out perfectly, since you’d hit *(::AbstractArray, ::AbstractArray), which will be suboptimal but at least work.

The implied problem here of course is that all subtypes of AbstractArray need to perfectly adhere to the interface AbstractArray defines, which is not rigidly enforced by the compiler at the moment, nay, not even known to the compiler. So it couldn’t optimize anything inside of that “most generic method”, even if it wanted to. On top of that, there’s no concept for this kind of specific “specialize this specific method more” concept, so that would have to be invented too. Maybe that’s worth it, maybe it isn’t, I don’t know :person_shrugging:

But this thread is about multiple inheritance. So instead of one AbstractArray you instead have a tangled soup of different traits that might apply to a given array. There’s not necessarily one common supertrait anymore once you start doing that.

1 Like

Yes, and that’s the least-common-supertype Any worstcase that has to be disambiguated manually in any case that I mentioned above.

I don’t see how “multiple abstract inheritance/subtyping” suddenly means that we don’t have any hierarchy between trait-types at all anymore; nothing stands in the way of having abstract trait-types that subtype AbstractArray, and individual concrete struct types picking multiple of those AbstractArray-traits-types to actually subtype.

The only example I can come up with in your scenario would be some type that is both <: AbstractArray as well as <: AbstractString (as an example where the least common supertype is Any)? I mean, sure, that could happen, but that seems incredibly unlikely to me, as well as screaming “programmer error” since defining a method handling that kind of case would explicitly go against “don’t implicitly vectorize stuff”. My gut feeling is that most cases that are kind of like that would be better served by a using different function at the callsite for whatever you’re trying to do entirely, which would also resolve the ambiguity. Or invoke at the callsite with the intended signature :person_shrugging: