I think the interesting question is not so much in which ways ambiguities could happen, but rather if they actually would in real code, and to which extent they would present a significant issue for the ecosystem.
In particular, these toy examples are certainly useful to demonstrate the prior but do not really capture the latter.
I think of an interface as a relatively small entitiy, containing only a small number of functions that focus on a single thing, much like the advise to keep functions small and focussed on doing a single thing.
Given that, it doesn’t make much sense at all to implement multiple interfaces with a common supertype. Or put differently - the ambiguity involved is of semantic nature, and the user simply has to choose which version of the “single thing” he wants.
For example, it might not be super meaningful to implement two interfaces that customize iteration in a different manner.
Maybe multiple inheritance is also the most useful when implementing interfaces representing really basic concepts, and higher up the type ladder a shift towards single inheritance make sense.
But ultimately we will only know for sure if we try.
And I feel like multiple inheritance holds way too much promise to not try!
I hinted at it in my earlier comparison, but this question applies just as much to multiple dispatch introducing method ambiguities that single dispatch doesn’t. Some of the documentation’s suggestions are patterns that can be done with single dispatch, which is evidently enough for many things. Add the avoidance of type piracy (which I assumed in my examples so it made package composability look a lot safer than it could be), and the language is usable and packages can work together with reasonable effort. Principles demonstrated by toy examples are exactly how we prevent chaos, and I think we already have a decent idea of the principles for required and provided (@idispatch) methods. With some elaboration (I think it should be stricter at some points, but don’t take my word on this), it should just work together with pre-existing multiple dispatch principles to avoid the expanded ambiguities. Multimethods can actually help a lot in that respect.
The number of arguments during method dispatch is fixed, whether at compile-time or runtime, so methods with different arity aren’t a concern. Two interfaces that share a required method name would easily avoid conflicts (e.g. one requires foo(::X, ::Integer)::String and the other requires foo(::X, ::Int)::Symbol) if the methods had different arity. Obviously MultipleInterfaces.jl doesn’t specify signatures of required methods at the moment, but we can figure out what we want in practice and request features.
We can split an @interface into mutually exclusive subinterfaces (say Indexable into IndexLinear and IndexCartesian) for easy Holy trait-like @idispatch, although the type-centered @interface concept can’t map multiple input types to 1 trait like a Holy trait-set function. All the trouble of Mouse and the other rodent types came from trying to implement some combination of B, C, D, etc, so that wouldn’t happen if B, C, D were mutually exclusive subinterfaces of A (this superinterface wasn’t present to make such a distinction in my first @type M implements B, D example). I think other packages can safely make their own mutually exclusive subinterfaces and associated provided methods, but I’d rather someone check that.
I agree with @_bernhard. Benny’s analysis of possible ambiguities is correct, but every design decision involves tradeoffs. The question is whether the benefits outweigh the costs. We can philosophize all day long, but the real question is whether ambiguities will be a significant issue in the real world. As I mentioned above, I’m hopeful that ambiguities won’t be too common, because:
Required methods won’t have ambiguities, because they dispatch on concrete types.
Provided methods usually only have a small number of specializations, and those specializations are often controlled by the function owner.
In addition to those reasons, there’s also the reason that @_bernhard mentioned:
It might not be very common for types to implement two or more closely related interfaces, like B and C from the example above, where B and C are different extensions of the A interface. I think it’s more common for types to implement orthogonal interfaces, like this:
@type Foo implements Iterator, Show, Serializable
MultipleInterfaces.jl might not get much adoption in the ecosystem, but if it does we’ll finally have some hard data on ambiguities.
In some ways, ambiguities are better than what we have right now:
We usually want to dispatch functions based on the behavior of their arguments, but abstract types are only a rough proxy of behavior, since they cannot capture the behavior of objects that implement multiple interfaces.
It’s not even possible right now to extend an informal interface and then provide a specialization of a provided method, unless you engage in type piracy.
Let me provide an example to illustrate the second point:
PackageA provides an informal interface with the required method a and a provided method foo.
The interface is informal and does not have an abstract type associated with it (similar to, e.g., iteration), so the signature of foo is foo(x::Any).
PackageB extends the informal interface in PackageA with the required method b.
In order to specialize foo for the extended interface, PackageB will have to overwrite the foo(x::Any) method, which is type piracy.
PackageC extends the informal interface in PackageA with the required method c.
In order to specialize foo for the extended interface, PackageC will have to overwrite the foo(x::Any) method, which is type piracy.
So, in order to extend the provided method foo, both PackageB and PackageC have engaged in type piracy. And, as usual with type piracy, if a user tries to use both PackageB and PackageC at the same time, then the particular definition of foo that gets used will depend on the order in which PackageB and PackageC are loaded.
We could promote the following design rule of thumb, which would eliminate most, if not all, ambiguities:
If you extend an interface from another package, you can add new provided methods, but try to avoid adding specializations to existing provided methods.
I don’t know yet if it’s a reasonable restriction, but it’s worth thinking about.
I did consider adding a macro to MultipleInterfaces.jl that would allow the user to choose which i-method gets dispatched to, if they need to resolve an ambiguity. (In the docs, I refer to the methods created with @idispatch as “i-methods”.) Something like this:
@invoke_imethod foo(Mouse(): B)
I decided not to add it, at least for now, because I thought that users usually won’t know which i-method is better to dispatch to. In the example above, how will the user know whether to dispatch to foo(x: B) or foo(x: C)? Although, if package authors have correctly provided specializations for foo, i.e. foo retains the same generic definition, then the user should get back the right answer regardless of which method they dispatch to. So, it’s ok for the user to just pick one at random…
So, a macro like @invoke_imethod is an option on the table if we really need it.
That would be true if the concrete type annotation is on the same positional argument, which is the leading argument in other languages because of obj.method(arg2, ...) syntax and method encapsulation by typeof(obj) in some block. That’s not the case for multimethods, so it is actually possible to run into method ambiguities, even if we’re not implementing a type for 2 interfaces at once:
# initial work in progress
abstract type AbstractRecipient end
mutable struct Recipient<:AbstractRecipient message end
setmessage!(x::Recipient, v) = (x.message = v; x)
abstract type AbstractSender end
struct Sender<:AbstractSender message end
getmessage(x::Sender) = x.message
function send! end #TODO: implement this
# contributor 1 tries to generalize Recipient
using MultipleInterfaces
@interface Receive begin
send! # send!(_, x::AbstractSender)
end
@type Recipient implements Receive
send!(y::Recipient, x::AbstractSender) = (y.message = getmessage(x); y)
# contributor 2 tries to generalize Sender
using MultipleInterfaces
@interface Send begin
send! # send!(x::AbstractRecipient, _)
end
@type Sender implements Send
send!(x::AbstractRecipient, y::Sender) = setmessage!(x, y.message)
# test fails
send!(Recipient(""), Sender("hello!")) # ambiguity
Conflicting interface annotations in @idispatch required methods can also cause an ambiguity, but I can’t make an example because @idispatch as of v1.0.0 can only annotate types by name in the scope of MultipleInterfaces instead of arbitrary expressions evaluating to types in the macro call scope, which I assume is a bug. Not sure if @idispatch methods shouldn’t be required methods, but something similar is allowed in other languages at least. It’s also worth noting that ambiguity isn’t the only bad consequence; if the 2 send! methods ended up with the same or more specific signatures instead, then the interfaces overwrite each other’s methods (not allowed in precompilation) or interfere with each other in unintended ways during dispatch.
Not really sure how that will go.
Informal interfaces so far don’t specify that required methods are implemented for the first argument (they don’t even specify that the overall interface must be centered on 1 type), but that could be a reasonable restriction for a more formal @interface (e.g. send! would not belong to @interface Send). What gives me pause is that input functions are often the first argument to allow do-blocks, and we wouldn’t be implementing an interface for that argument. (Aside: many methods in informal interfaces are actually optional and fall back to derivative methods or MethodErrors, so maybe a different term like “interface method” is warranted.)
One thing in Julia’s favor is that generic functions usually have module-encapsulated names, so one very reasonable practice is to use different functions across different interfaces even if they happen to share a name. Barring retroactively formalized interfaces like for Base, modules should introduce interfaces and their required functions together, and those required functions would be indirectly adopted by other interfaces by extending interfaces. In that case, it’d be very useful to be able to discover a function’s original interface given loaded modules (e.g. findinterfaces(iterate) == Iterable).
Unless something ends up contradicting this, it does appear that users can safely write provided methods for mutually exclusive subinterfaces (that is, each type can only implement one of these), equivalent to adding a new subtype to a Holy trait set and writing methods for it. Given how Holy traits is currently the more typical approach to multiple inheritance of types (but single inheritance per contextual provided method), relaxing the new “interface piracy” concept could be very worth it.
The A superinterface wasn’t necessary for ambiguity in your example, and in my f, B, D, M example, B and D were intended to be completely separate interfaces annotated in different methods of f. A type trying to implement multiple interfaces and those interfaces annotating different methods in the same provided @idispatch function were enough. In this comment’s example, the types Sender and Recipient and the interfaces Send and Receive were also distinct, and the ambiguity was caused by sharing a required function with conflicting method signatures.
mmm but can i rely on them being types? so that instead of dispatching on the singleton object (e.g. f(::InterfaceA)), i dispatch on the type (e.g. f(::Type{InterfaceA})).
Regarding integration, I think I could refactor my DelegatorTrait as a Delegated{Interface} interface. In this way I could easily do…
@interface Delegated{T} begin
delegator
end
@idispatch my_function(x: Delegated{MyInterface}) = my_function(delegator(MyInterface, x))
Since I guess that interfaces with type-parameters are not supported, I could replace Delegated{MyInterface} for MyInterfaceDelegatable and have a macro autogenerate them.
The ultimate goal is that something like this works:
struct WrappedIterator{T}
it::T
end
@delegate Iterator WrappedIterator{T} to it
That looks to me like the usual multi-argument ambiguity that we’ve had in Julia since it was created. In this discussion, I’ve been focusing on the new ambiguities that multiple inheritance creates, i.e. single argument ambiguities.
Regarding your example, it seems more natural to me to make send! a provided method. So, using the MultipleInterfaces.jl syntax, that would look like this:
function getmessage end
function setmessage! end
@interface Sender begin
getmessage
end
@interface Reciever begin
setmessage!
end
@idispatch send!(a: Reciever, b: Sender) = setmessage!(a, getmessage(b))
I haven’t yet seen an example of a required method that really needs to belong to two independent interfaces, though there very well could be one. (Where “independent” means that neither is a subinterface of the other.) Though I haven’t looked too hard yet.
"""
add_edge!(g, e)
Add an edge to a graph.
- `g` implements the `MutableGraph` interface
- `e` implements the `DirectedEdge` interface
"""
@interface MutableGraph begin
add_edge!
# ...
end
Both arguments in the docstring are described in terms of the interface that they implement, but it’s fairly clear in this example that add_edge! should be a required method of the MutableGraph interface, not the DirectedEdge interface.
Here’s another example. Consider a hypothetical fit method for a machine learning model interface:
"""
fit(model, X, y::AbstractVector)
Fit a machine learning model.
- `model` implements the `Model` interface
- `X` implements the `Table` interface
"""
In this example, it wouldn’t make sense for fit to be a required method of the Table interface. It’s only a required method for the Model interface.
To be clear, MultipleInterfaces.jl does not care which argument of a required method expects to receive an implementor of the interface under consideration. It doesn’t have to be the first argument.
It often would make sense to use @idispatch to implement multi-argument required methods. I gave this example further up in the thread:
So you can see in this example that the argument of the required method that expects an implementor of the interface dispatches on a concrete type, while the other argument dispatches on an interface. (In this case MyConcreteGraph is the implementor of the Graph interface.)
I agree. I’m planning to expand the documentation and create a Documenter.jl website, and I’m planning to discuss guidelines for and examples of defining and implementing interfaces. “Interface piracy” would be one of the points discussed (it is discouraged, just like with type piracy).
Yes, when I introduced my version of the example with a common superinterface A, I mentioned that the common superinterface was not necessary to generate the ambiguity. But it is conceptually important because it illustrates the “right way” to develop provided methods:
Any generic function (including provided methods) should have a single docstring with a single generic definition (or one for each arity of the function).
The interfaces that each argument are expected to implement should be documented.
So, for example, the definition of foo in PackageA might look like this:
"""
foo(x)
Do a foo thing with `x`.
- `x` implements the `A` interface
"""
@idispatch foo(x: A) = # ...
A consequence of this principle is that any new, single-argument methods of foo must dispatch on a subinterface of A, because foo is documented to only work with objects that implement the A interface. Thus, B and C cannot be “completely separate interfaces”. That would violate a very important principle of generic programming. Thus, single-argument ambiguities in provided methods will only occur if a type implements two or more very closely related interfaces. (Assuming folks do a good job of following generic programming principles. )
Are you referring to the fact that the following does not work?
julia> using MultipleInterfaces
julia> function a end;
julia> function b end;
julia> @interface A begin a end
julia> @interface B begin b end
julia> foo(flag) = flag ? A : B;
julia> @idispatch bar(x: foo(true)) = 1
ERROR: LoadError: Syntax error in the `@idispatch` macro.
That’s technically not a bug, because the documentation never says that arbitrary expressions like that should work. But if you think it would be a useful feature, please open a Github issue.
Yes, the fourth line of the following example is an implementation detail, but it is very likely to remain true:
julia> using MultipleInterfaces
julia> function a end;
julia> @interface A begin a end
julia> A isa Type
true
I’m not sure if you need to worry about interface intersections for you application. If so, that could be a little trickier. The type of the intersection A & B is Intersection{Tuple{A, B}}. And interface intersections are normalized by lexicographically sorting the interface names. (The parent module names are included in the sorting.)
Right, interface parameters are currently not supported. My current thinking is that type/interface parameters are an implementation detail. If users of an interface need a way to interact with some kind of element type, then the interface should include a required method to do that, like eltype, keytype, valtype, vertex_type, or edge_type.
I think it’s mostly “allowed” in other languages just because it’s impossible to prevent two independent developers from writing the same method name in different interfaces, and just saying the interfaces were incompatible isn’t an option. But it’s deliberately avoided, and those languages can’t additionally distinguish those names by parent module even if they never intended those names to overlap in behavior.
That’s just the arbitrary expression bit, and I honestly wouldn’t mind if it were disallowed because not involving the interface by name in the provided method definition is just weirdly opaque, especially given the extra considerations for extensions over other generic methods. Omitting that, the rest means this:
julia> using MultipleInterfaces
julia> function b end; @interface B begin b end
julia> struct Capybara end; @type Capybara implements B
julia> @idispatch foo(x: B, y::String) = 0 # documented usage
foo (generic function with 1 method)
julia> @idispatch foo(x: B, y::Capybara) = 1 # why search for a ::name there?
ERROR: UndefVarError: `Capybara` not defined in `MultipleInterfaces`
It’s corroborated by the expanded expressions:
julia> @macroexpand @idispatch foo(x: B, y::String) = 0
quote
if !(MultipleInterfaces.isdefined(Main, :foo)) || !(MultipleInterfaces.is_signature_defined(foo, (MultipleInterfaces.InterfaceArg, MultipleInterfaces.String)))
function foo(var"#129#x", var"#130#y"::MultipleInterfaces.String)
...
end
julia> @macroexpand @idispatch foo(x: B, y::Capybara) = 0
quote
if !(MultipleInterfaces.isdefined(Main, :foo)) || !(MultipleInterfaces.is_signature_defined(foo, (MultipleInterfaces.InterfaceArg, MultipleInterfaces.Capybara)))
function foo(var"#107#x", var"#108#y"::MultipleInterfaces.Capybara)
...
end
I think parameters/generics are mostly for method signatures, which @interface does not currently support. Otherwise, it would be useful to @idispatch over an interface sharing a parameter with the type and other arguments, like @idispatch push!(A::Indexable{T}, i::T} where T accepting A::Vector{Int}, i::Int, or as a parameter @idispatch feed!(A::Vector{T}) where {T implements B} accepting A::Vector{Capybara} and maybe even A::Vector{Union{CapyBara, Rat, Jerboa}}. I don’t expect a full integration of interfaces into the supertype system to actually instantiate Vector{T} where {T implements B} or a concrete Vector{implements B}.
)