# How is the type of a heterogeneous vector determined when splatting and doing list comprehension?

Consider the type of the resulting `Vector` in these three cases:

``````[1, 1.0]  # case 1: Vector{Float64}, elements are promoted
v = Any[1, 1.0]
[v...]  # case 2: gives identical result to case 1
[x for x in v]  # case 3: Vector{Real}, elements are not promoted
``````

My first question is, what is the logic behind case 3 being different from case 1 and case 2?

In any case, I would like to replicate behavior analogous to this with custom types. I was able to reproduce the splatting behavior (case 1/2), by implementing methods for `promote_rule` and `convert`, but I do not know how to achieve the behaivor analogous to case 3 for list comprehension.

Let me walk through an example implementing the desired promotion for splatting (but failing for list comprehension):

``````struct Foo{T}
x::T
end

f1 = Foo(1)
f2 = Foo(1.0)
``````

Note that at this stage these three cases all give the same `Vector{Foo}`:

``````[f1, f2]  # case1
[Any[f1, f2]...] # case 2
[x for x in Any[f1, f2]]  # case 3
``````

However, the desired behavior, in analogy with the first `Number` example, is that the first two cases should give a `Vector{Foo{Float64}}` by promoting the elements, and the last case should give a `Vector{Foo{Real}}` leaving the elements untouched. To achieve the right behavior for case 1 & 2, I implement `promote_rule` and `convert` methods (following examples given in the docs):

``````Base.promote_rule(::Type{Foo{T}}, ::Type{Foo{S}}) where {T, S} =
Foo{promote_type(T, S)}
Base.convert(::Type{F}, f::Foo) where {F <: Foo} = F(f)

Foo{T}(f::Foo) where {T} = Foo{T}(f.x)
``````

Now case 1 and case 2 correctly promotes the elements to `Vector{Foo{Float64}}`, however, case 3 gives an error

``````ERROR: MethodError: Cannot `convert` an object of type Int64 to an object of type Foo{Int64}
``````

This error came as a surprise. Why does it end up attempting to convert an `Int` to a `Foo` here? How do I get the desired result of case 3 resulting in a `Vector{Foo{Real}}` with elements `Foo{Int64}(1), Foo{Float64}(1.0)` in analogy with `Number`s?

1 Like

The stacktrace is informative here, and the cause of the trouble is the call to `convert`, combined with the method `Foo{T}(f::Foo) where {T} = Foo{T}(f.x)`:

``````julia> convert(Foo, f1)
ERROR: MethodError: Cannot `convert` an object of type Int64 to an object of type Foo{Int64}
Closest candidates are:
convert(::Type{F}, ::Foo) where F<:Foo at REPL:1
convert(::Type{T}, ::T) where T at ~/programs/julia/julia-1.7.0/share/julia/base/essentials.jl:218
Foo{T}(::Any) where T at REPL:2
Stacktrace:
 Foo{Foo{Int64}}(x::Int64)
@ Main ./REPL:2
 Foo{Foo{Int64}}(f::Foo{Int64})
@ Main ./REPL:1
 Foo(x::Foo{Int64})
@ Main ./REPL:2
 convert(#unused#::Type{Foo}, f::Foo{Int64})
@ Main ./REPL:1
 top-level scope
@ REPL:1
``````

The way I would remedy the situation is by removing the `Foo{T}(f::Foo)` method and then changing the `convert` method to

``````Base.convert(::Type{Foo{T}}, f::Foo) where {T} = Foo(convert(T, f.x))
``````
1 Like

Thanks Steven. That indeed leads to running code for all three cases. With that solution,“case 3”,

``````[x for x in Any[f1, f2]]
``````

gives a `Vector{Foo}` which is the default behavior. I wonder how to get this to become a `Vector{Foo{Real}}` though? Is there some other function than `promote_rule` that has to have a method, something that specifies the first common supertype or something like that? It’s still opaque to me what happens under the hood for “case 3”.

The reason why the result is a `Vector{Foo}` is because `Foo` is the first common supertype of `Foo{Int64}` and `Foo{Float64}`. Importantly, `Foo{Real}` is a concrete type, and thus

``````julia> Foo{Int64} <: Foo{Real}
false
``````

so a `Vector{Foo{Real}}` would have to be constructed with objects that are all of type `Foo{Real}`.

1 Like

Not quite; `Foo{<:Real}` is the first common supertype.

``````julia> (Foo{Int}, Foo{Float64}) .<: Foo{T} where T<:Real
(true, true)

julia> Foo{<:Real}[Foo(1), Foo(1.0)]
2-element Vector{Foo{<:Real}}:
Foo{Int64}(1)
Foo{Float64}(1.0)
``````

That `Foo{Int64} <: Foo{Real}` is `false` always trips me up. Would it make sense for `[x for x in Any[f1, f2]]` to return a `Vector{Foo{<:Real}}`, and if so, what method(s) would I need to overload to make that happen?

This seems to do the trick:

``````julia> struct Foo{T}
x::T
end

julia> Base.promote_rule(::Type{Foo{T}}, ::Type{Foo{S}}) where {T, S} =
Foo{promote_type(T, S)}

julia> Base.convert(::Type{Foo{T}}, f::Foo) where {T} = Foo(convert(T, f.x))

julia> Base.typejoin(::Type{Foo{T1}}, ::Type{Foo{T2}}) where {T1,T2} = Foo{T} where {T<:typejoin(T1, T2)}

julia> [x for x ∈ Any[Foo(1), Foo(1.0)]]
2-element Vector{Foo{T} where T<:Real}:
Foo{Int64}(1)
Foo{Float64}(1.0)
``````
3 Likes

Not quite; `Foo{<:Union{Int,Float64}}` is the first common supertype The point is, `Foo` is the only explicitly defined type in the tree, and what somebody means by “first common supertype” isn’t necessarily clear. For example, given `Foo{A}` and `Foo{B}`, how would Julia know to take the `Union` of `A` and `B` vs traversing the type trees of `A` and `B` to find a common supertype vs just focusing on the type tree of `Foo`? So I would argue that in some sense `Foo` is the first common supertype (of types explicitly defined in the type tree of `Foo`), and that developers have to tell Julia otherwise if they want a different definition of first common supertype, which seems to be accomplished with `typejoin` (which I didn’t know about until today).

1 Like

I stand corrected. I also didn’t know about `typejoin` which solves the problem!

One small issue with the proposed solution though is that `convert(Foo{T}, f::Foo)` isn’t guaranteed to return a value of type `Foo{T}`. E.g. `convert(Foo{Any}, Foo(1))` returns `Foo{Int64}(1)` which is not a `Foo{Any}`. So I suppose the correct thing would be to define `Base.convert(::Type{Foo{T}}, f::Foo) where {T} = Foo{T}(f.x)`

1 Like