Method specificity gotcha

Internals & Design
1

I just spent a lot of time debugging what came down to the following:

struct Foo{T<:Integer} end

f(::Type{F}) where {T, F<:Foo{T}} = 1
f(::Type{F}) where {F<:Foo} = 2

I expected f(Foo{Int}) to return 1 and f(Foo) to return 2, yet

julia> f(Foo{Int})
2

julia> f(Foo)
2

If you remove the <:Integer constraint from the definition of Foo, f(Foo{Int}) returns 1 as expected. Alternatively, replacing the first method of f with

f(::Type{F}) where {T<:Integer, F<:Foo{T}} = 1

also works as I expected.

Is this behavior desired? It was certainly surprising to me.

2 Likes
2

Seems relevant:

julia> struct Foo{T<:Integer} end

julia> Foo <: (Foo{T} where T)
true

julia> (Foo{T} where T) <: Foo
false

julia> Foo <: (Foo{T} where T <: Integer)
true

julia> (Foo{T} where T <: Integer) <: Foo
true

julia> struct Foo{T} end

julia> Foo <: (Foo{T} where T)
true

julia> (Foo{T} where T) <: Foo
true

May I ask why is this expected? I think if you remove <:Integer, the 2 functions become semantically indistinguishable, not sure why Julia goes for one over the other, perhaps some subtle rule in the code?

1 Like
3

I mostly expected this from previous experience. Unfortunately, the documentation on method specificity could be better, Wishlist: document method specificity rules · Issue #23740 · JuliaLang/julia · GitHub. I do think it’s desired to have this behavior, since it can be quite useful to distinguish between f(Foo) and f(Foo{SomeT}) for all SomeT without having to define a method for each possible SomeT: If f is called with e.g., Foo{String} or Foo{Int} as the argument, the user has supplied more information than when f is called with just Foo as the argument, and it should be possible to distinguish between these two cases using appropriate method definitions.

I think there are two options to make the behavior from my first post less surprising.

Option 1

The first option would be to have the first method

f(::Type{F}) where {T, F<:Foo{T}} = 1

be automatically interpreted as

f(::Type{F}) where {T<:Integer, F<:Foo{T}} = 1

i.e., to deduce from the parameter bounds of Foo that T must always be <:Integer for the method to make sense. In general if T is a parameter of more than one type, perhaps this would need to be done using typeintersect.

Option 2

The other way would be to interpret the second signature,

f(::Type{F}) where {F<:Foo} = 2

so that it doesn’t have the implicit <:Integer constraint (due to the fact that Foo === Foo{T} where T<:Integer) and acts like

f(::Type{F}) where {F<:Foo{T} where T} = 2

(which is another way to get the behavior I expected).

I think option 1 would be more desirable (because the truth value of Foo === Foo{T} where T<:Integer should probably not be context-dependent). I suspect there may be practical reasons that make option 1 infeasible though, namely that typeintersect(T, S) is not guaranteed to produce the ‘smallest’ R such that R<:T and R<:S.

I guess option 3 would be to just acknowledge that this is a bit of a corner case and accept that you need to be more careful in such cases.

1 Like
4

This is a longstanding issue: https://github.com/JuliaLang/julia/issues/6383

I hope to try to implement your “option 1”; coming up with a way to automatically propagate the variable bounds is just tricky.

3 Likes
5

Good to hear! Thanks for the pointer.