Question on manual subsection: Extracting the type parameter from a super-type

New to Julia
, ,
1

Question:
One correct and two wrong code templates for extracting the type parameter from a super-type is provided in this subsection of the manual. I don’t understand how the “correct template” doesn’t fail for the argument that is given as an example where the (second) “wrong template” might fail, and would appreciate if someone can help me grasp it. Thank you!

Details:
(From the manual subsection) The abstract type definition and the correct template is:

abstract type AbstractArray{T, N} end
eltype(::Type{<:AbstractArray{T}}) where {T} = T

And the (second) wrong template is:

eltype_wrong(::Type{AbstractArray{T}}) where {T} = T
eltype_wrong(::Type{AbstractArray{T, N}}) where {T, N} = T
eltype_wrong(::Type{A}) where {A<:AbstractArray} = eltype_wrong(supertype(A))

Now the manual gives an example where this wrong approach fails:

julia> eltype_wrong(Union{AbstractArray{Int}, AbstractArray{Float64}})
ERROR: MethodError: no method matching supertype(::Type{Union{AbstractArray{Float64,N} where N, AbstractArray{Int64,N} where N}})
Closest candidates are:
  supertype(::DataType) at operators.jl:43
  supertype(::UnionAll) at operators.jl:48

but also the “correct” code template fails when I call it with the same argument (or falls back to Base.eltype, but the same holds if we rename the “wrong” code template):

julia> eltype(Union{AbstractArray{Int}, AbstractArray{Float64}})
ERROR: MethodError: no method matching eltype(::Type{Union{AbstractArray{Float64}, AbstractArray{Int64}}})
You may have intended to import Base.eltype
Closest candidates are:
  eltype(::Type{<:AbstractArray{T}}) where T at REPL[9]:1
Stacktrace:
 [1] top-level scope
   @ REPL[31]:1

It doesn’t say explicitly that the “correct template” should work for these arguments, but since it is given to justify why the alternative is wrong, I take it as implied. And of course there are other reasons why the “correct template” is preferred, but I would like to understand the reason given in the manual.

2

On a fresh julia 1.8 session

julia> eltype(Union{AbstractArray{Int}, AbstractArray{Float64}})
Any
julia> @which eltype(Union{AbstractArray{Int}, AbstractArray{Float64}})
eltype(::Type) in Base at abstractarray.jl:204

The error You may have intended to import Base.eltype that you may be trying to redefine some method inadvertently. If not, then something is off.

3

Thank you @raminammour, but that’s exactly my point. The fallback method works fine, as stated in the manual subsection I am referencing:

… The implementation of eltype in Base adds a fallback method to Any for such cases.

but we could as well have written an extra method in the “wrong” approach that is more specific than eltype_wrong(::Type{A}) where {A<:AbstractArray}, so that it will dispatch on such non-singular Union subtypes of AbstractArray? Or is it not possible? Granted, in any case it is much simpler to dispatch on the intended cases as the more specific method and clearly better design by any measure. But then it’s not about the limitations of the wrong approach, as indicated, but that:

  • The correct approach dispatches exactly on the relevant types for the desired behavior and leaves ample room so that fallback method can be defined very generally, i.e. eltype(::Type) = Any
  • The wrong approach, as it is, dispatches on more than the relevant types, so that instead of a simple general fallback method, one needs to “weed out” the irrelevant types with more specific methods to replicate the “fallback” behavior.

Would this be a fair summary?

If so, I think the manual subsection would be much clearer if the fallback is explicitly written for the eltype, as part of the “correct code template”, i.e:

abstract type AbstractArray{T, N} end
eltype(::Type) = Any
eltype(::Type{<:AbstractArray{T}}) where {T} = T

And looking at the source code, even this approach needs an additional method to weed out “empty” Union type:

eltype(::Type{Bottom}) = throw(ArgumentError("Union{} does not have elements"))
4

I think the manual has a bad example. Before I explain, I think the discussion is getting a little confused; the manual is showing hypothetical source code. If you run it in your own module or REPL, those scopes are accessing your own personal AbstractArray and eltype definitions, not the ones in Base. Avoid the types, and rename functions if you want to try them out, e.g. eltype to eltype_correct.

Back to the point, the manual is trying to make a distinction between Type{<:AbstractArray{T}} and Type{AbstractArray{T}} in the argument annotations. The former specifies arguments being any subtype of AbstractArray, like Array{Int, N} where N, but the latter only specifies AbstractArray itself. T being in the method’s where clause means the inputs must provide a specific type for T.

The supertype-using eltype_wrong method does work on any subtype of AbstractArray, but it does not specify the parameter T we want; instead it tries to recursively supertype its way to AbstractArray so the other eltype_wrong methods can take it. It is indeed correct that if supertype doesn’t work, eltype_wrong doesn’t work.

However, the correct code as is wouldn’t work either because Union{AbstractArray{Int}, AbstractArray{Float64}} does not have a specific type for T, despite being a subtype of AbstractArray. The example has an easy fix, make the types in the Union share a type for T:

eltype_correct(Array{Int,3}) # Int
eltype_correct(Union{Array{Int,1}, Array{Int,3}}) # Int
eltype_wrong(Array{Int,3})   # Int
eltype_wrong(Union{Array{Int,1}, Array{Int,3}})   # errors

Incidentally, this is where it is important to specify parameters belonging to the argument versus the method. A version of the correct code for the number of dimensions would look like

my_ndims(::Type{<:AbstractArray{T,N} where T}) where {N} = N
my_ndims(Union{Array{Int,1}, Array{Bool,1}}) # 1

Base.ndims now behaves like this, but it used to error on unspecific T like this Union or Array{T,1} where T because its method had both parameters {T,N} (issue #40682).

2 Likes
5

Very enlightening, thank you @Benny for clear and detailed explanation! (additional edits were also immensely helpful)

It will indeed take some practice to really master the leveraging the power of Julia’s type space, since knowing \neq applying. Takeaways for me are:

  • Leveraging MyType{T} <: MyType and Union{MyType{T1}, MyType{T2}} <: MyType for T1, T2 (or more) are valid (i.e. respects the constraints of MyType) in order to catch the appropriate sub-types.
  • Be mindful about MyType{T, S} <: MyType{T} <: MyType and MyType{T, S} <: MyType{S} <: MyType and “parameters specified in the argument vs the method” to catch the appropriate sub-types.
  • supertype operation and <: operation shouldn’t be used interchangeably. For example:
Union{Array{Int, 1}, Array{Int, 3}} <: Array  # true
supertype(Union{Array{Int, 1}, Array{Int, 3}})  # errors
1 Like