Interfaces/traits in julia 2.0 and multiple inheritance

[Janis_Erdmanis]

I will take an ignorant stab at this, but I believe what we really need is more powerful unions that can be put into function signatures and extended afterwards. For instance, if we were able to do a simple thing like:

const FieldTrait = Union{<:Integer} # Which could be defined elsewhere

f(x::FieldTrait) = 2*x

struct MyField
    x
end
const FieldTrait = Union{MyField, FieldTrait}

where the function f would reference the new union there would be no need for a trait system in Julia. The trait constraint then becomes intersection of unions which is easally to grasp and does not require any new additions to dispatch logic and reasoning how it works out.

For me the key deterent for trying to incorporate any of the trait libraries is that they often require the use of macros for the function definition which obscures debugging. Even more so when having had experiences of debugging dispatch ambiguities and trying to figure out how to resolve them constructively.

[serhii]

@Janis_Erdmanis

where the function f would reference the new union there would be no need for a trait system in Julia. The trait constraint then becomes intersection of unions which is easally to grasp and does not require any new additions to dispatch logic and reasoning how it works out.

This is already possible in julia with UnionAll type

function f(x::T) where T <: Integer
    2*x
end

struct MyField <: Integer
    x
end

Base.:*(x::MyField, y::Integer) = MyField(x.x * y)
Base.:*(x::Integer, y::MyField) = MyField(x * y.x)

f(MyField(2))

T where T <: Integer is essentially the same as Union{<:Integer}

julia> (T where T <: Integer) === (Union{<:Integer})
true

const FieldTrait = Union{MyField, FieldTrait} semantically is the same as struct MyField <: Integer

[cdawg]

I love these discussions, even though I believe I can feel the mods and people writing PRs to base rolling their eyes: “Not this again”

All I know is the problems I encounter and what it feels like I am missing, with little understanding of how added features will wreak havoc. I have not tried to write a sparse library.

From this vantage, it seems like multiple inheritance/traits are not a big improvement for me, but I would like concrete subtyping. The stupid way I see this working is simply make T <: Abstract{T} so that there is just this automatic extra node in the type graph. The one tiny change is that all function signatures x::T get lowered to x<:Abstract{T}

Then if you want to subtype S <: :Abstract{T} you have to make sure that you have overloaded getfield(x::S,:fieldnameofT). Bam. Concrete subtyping. Problem solved? Am I a moron or a genius?

And yeah, just like public, we just need a base-approved way to name interfaces, kind of an orthogonal issue though but the most obvious interface is this kind of “shadow” implementation of another concrete subtype.

[Janis_Erdmanis]

I am just illustrating how to not commit fully on multiple inheritance while enabling expermentation on different trait system designs. What I would find more realistic is that one would create a function for appending a type to the union which runs various checks on implemented interface. Probably it would converge to something but at the moment it is hard to tell.

[nsajko]

Keno has a document on a possible interfaces design: A roadmap for interfaces - HackMD

[CameronBieganek]

I’m currently working on a package called ExtendableInterfaces.jl that provides multiple inheritance among interfaces, and multiple dispatch on interfaces. (Hopefully I can find some time during the holidays to get it over the finish line. Though I also have to write the documentation… ) My hope is that if users develop interfaces and generic functions with the right design principles, then method ambiguities will be manageable. However, we won’t really know if multiple inheritance is feasible until a decent subset of the ecosystem tries it out in practice.

Let’s consider an example similar to the one provided by @Mason. We only need single-argument functions to demonstrate the ambiguities that can arise with multiple-inheritance, so I will stick with single-argument functions for this example. The syntax used is that of ExtendableInterfaces.jl, but I think it should be fairly intuitive to read.

There are four packages in this example:

Pkg1

function a end

@interface A begin
    a
end

"""
    foo(x: A)

Foo-inate an `A`.
"""
@idispatch foo(x: A) = (a(x), 42)

Pkg2

using Pkg1: Pkg1, A, a, foo

function b end

@interface B extends A begin
    b
end

@idispatch Pkg1.foo(x: B) = (b(x) * a(x), 42)

Pkg3

using Pkg1: Pkg1, A, a, foo

function c end

@interface C extends A begin
    c
end

@idispatch Pkg1.foo(x: C) = (c(x) * a(x), 42)

Pkg4

using Pkg1: foo
using Pkg2: B
using Pkg3: C

struct Cat end

@type Cat implements B, C

foo(Cat()) # ambiguity error

In this case, foo(Cat()) results in an ambiguity error. There’s nothing that Pkg4 can do about it (or nothing that they should do about it), because Pkg4 owns neither foo nor A, B, or C, so adding another @idispatch for foo would be interface piracy.

However, there is a rule of thumb for concrete types that I would propose:

• Avoid implementing two or more interfaces that have a common superinterface.
  • If you own at least one of the interfaces that you are implementing, then it’s probably not an issue, since you can define dispatches on interface intersections (denoted B & C in ExtendableInterfaces.jl).

That might sound restrictive, but I think in practice it’s often pretty natural. For example, it would be reasonable for a type to implement two orthogonal interfaces, Iterable and PushAndPoppable, like this:

struct Dog end
@type Dog implements Iterable, PushAndPoppable

There are some other interface and generic function design principles that collectively might help to reduce ambiguities. First, a few definitions:

• A required method is a method that must be implemented by a type as part of the requirements for an interface.
• A derived method is a method that is defined using the required methods for various interfaces (and possibly other derived methods).
• All functions in Julia are generic functions. (A single function can have multiple implementations.)

So, in the example above, the functions a, b, and c are required methods, and foo is a derived method.

Design principles:

• Required methods are normally only implemented for concrete types.
• Derived methods are implemented for abstract types (and for “interfaces” in ExtendableInterfaces.jl).
• Generic functions should have only one docstring per function arity. In other words, there should be exactly one defined behavior for each function arity.
  • There are some exceptions to this rule, like constructors and convert(T, x), where the exact behavior depends on the input types.

Note in the example above, Pkg1 provides a single, generic docstring for foo, and Pkg2 and Pkg3 do not add to the docstring, since they should not change the behavior of foo when they provide specializations for the subinterfaces B and C.

Finally, let’s consider a modified version of the above example where interfaces B and C are provided by the same package, Pkg2b. Then Pkg2b can use an interface intersection (denoted B & C) to disambiguate the dispatch for types that implement both B and C, like this:

Pkg2b

using Pkg1: Pkg1, A, a, foo

function b end
function c end

@interface B extends A begin
    b
end

@interface C extends A begin
    c
end

@idispatch Pkg1.foo(x: B) = (b(x) * a(x), 42)
@idispatch Pkg1.foo(x: C) = (c(x) * a(x), 42)
@idispatch Pkg1.foo(x: B & C) = (b(x) * c(x) * a(x), 42)

struct Chameleon end

@type Chameleon implements B, C

[CameronBieganek]

Let me expand a little on how the design principles reduce ambiguities.

If we follow the principle that a generic function should have only one docstring per arity (i.e. one behavior per arity), then the following ambiguity would be avoided:

function b end
function c end

@interface B begin
    b
end

@interface C begin
    c
end

@idispatch foo(x: B) = 1
@idispatch foo(x: C) = 2

struct Cat end

@type Cat implements B, C

foo(Cat())  # ambiguity error

Of course, the same kind of ambiguity can still occur when we do follow that design principle, as illustrated in the first example in my previous post (where the hierarchy is B ← A → C and the foo docstring is defined in terms of the interface A). However, if we also follow the rule of thumb that concrete types should avoid implementing two or more interfaces that have a common superinterface, then we also avoid the ambiguity in the B ← A → C case.

The rule of thumb might be closer to a requirement than a guideline, since users might be forced to adhere to it in order to avoid ambiguities…

Finally, there shouldn’t be ambiguities with interface required methods, since the design principle for required methods is that they should normally only dispatch on concrete types (or parametric types or unions that you own, etc). Granted, the large majority of functions used in the wild can be considered derived methods rather than required methods, but at least that’s one category of functions where we don’t need to worry about ambiguities.

All that being said, single inheritance combined with duck typing is already pretty powerful, as @jakobnissen mentioned, so it remains to be seen whether the benefits of multiple inheritance outweigh the costs.