# How is the type of a heterogeneous vector determined when splatting and doing list comprehension?

**URL:** <https://discourse.julialang.org/t/how-is-the-type-of-a-heterogeneous-vector-determined-when-splatting-and-doing-list-comprehension/92572>\
**Category:** General Usage\
**Created:** [January 5, 2023, 7:45pm UTC](https://discourse.julialang.org/t/how-is-the-type-of-a-heterogeneous-vector-determined-when-splatting-and-doing-list-comprehension/92572 "2023-01-05T19:45:04Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![arnelg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/arnelg/32/31530_2.png) [@arnelg](https://discourse.julialang.org/u/arnelg)\
**Post date:** [January 5, 2023, 7:45pm UTC](https://discourse.julialang.org/t/how-is-the-type-of-a-heterogeneous-vector-determined-when-splatting-and-doing-list-comprehension/92572/1 "2023-01-05T19:45:04Z")

</div>

Consider the type of the resulting `Vector` in these three cases:

```julia
[1, 1.0] # case 1: Vector{Float64}, elements are promoted
v = Any[1, 1.0]
[v...] # case 2: gives identical result to case 1
[x for x in v] # case 3: Vector{Real}, elements are not promoted

```

My first question is, what is the logic behind case 3 being different from case 1 and case 2?

In any case, I would like to replicate behavior analogous to this with custom types. I was able to reproduce the splatting behavior (case 1/2), by implementing methods for `promote_rule` and `convert`, but I do not know how to achieve the behaivor analogous to case 3 for list comprehension.

Let me walk through an example implementing the desired promotion for splatting (but failing for list comprehension):

```julia
struct Foo{T}
    x::T
end

f1 = Foo(1)
f2 = Foo(1.0)

```

Note that at this stage these three cases all give the same `Vector{Foo}`:

```julia
[f1, f2] # case1
[Any[f1, f2]...] # case 2
[x for x in Any[f1, f2]] # case 3

```

However, the desired behavior, in analogy with the first `Number` example, is that the first two cases should give a `Vector{Foo{Float64}}` by promoting the elements, and the last case should give a `Vector{Foo{Real}}` leaving the elements untouched. To achieve the right behavior for case 1 & 2, I implement `promote_rule` and `convert` methods (following examples given in the [docs](https://docs.julialang.org/en/v1/manual/conversion-and-promotion/)):

```julia
Base.promote_rule(::Type{Foo{T}}, ::Type{Foo{S}}) where {T, S} =
    Foo{promote_type(T, S)}
Base.convert(::Type{F}, f::Foo) where {F <: Foo} = F(f)

Foo{T}(f::Foo) where {T} = Foo{T}(f.x)

```

Now case 1 and case 2 correctly promotes the elements to `Vector{Foo{Float64}}`, however, case 3 gives an error

```julia
ERROR: MethodError: Cannot `convert` an object of type Int64 to an object of type Foo{Int64}

```

This error came as a surprise. Why does it end up attempting to convert an `Int` to a `Foo` here? How do I get the desired result of case 3 resulting in a `Vector{Foo{Real}}` with elements `Foo{Int64}(1), Foo{Float64}(1.0)` in analogy with `Number`s?

---

<div class="post-metadata">

**Author:** ![StevenWhitaker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevenwhitaker/32/9749_2.png) [@StevenWhitaker](https://discourse.julialang.org/u/StevenWhitaker)\
**Post date:** [January 6, 2023, 4:26pm UTC](https://discourse.julialang.org/t/how-is-the-type-of-a-heterogeneous-vector-determined-when-splatting-and-doing-list-comprehension/92572/2 "2023-01-06T16:26:30Z")

</div>

The stacktrace is informative here, and the cause of the trouble is the call to `convert`, combined with the method `Foo{T}(f::Foo) where {T} = Foo{T}(f.x)`:

```julia
julia> convert(Foo, f1)
ERROR: MethodError: Cannot `convert` an object of type Int64 to an object of type Foo{Int64}
Closest candidates are:
  convert(::Type{F}, ::Foo) where F<:Foo at REPL[17]:1
  convert(::Type{T}, ::T) where T at ~/programs/julia/julia-1.7.0/share/julia/base/essentials.jl:218
  Foo{T}(::Any) where T at REPL[11]:2
Stacktrace:
 [1] Foo{Foo{Int64}}(x::Int64)
   @ Main ./REPL[11]:2
 [2] Foo{Foo{Int64}}(f::Foo{Int64})
   @ Main ./REPL[20]:1
 [3] Foo(x::Foo{Int64})
   @ Main ./REPL[11]:2
 [4] convert(#unused#::Type{Foo}, f::Foo{Int64})
   @ Main ./REPL[17]:1
 [5] top-level scope
   @ REPL[32]:1

```

The way I would remedy the situation is by removing the `Foo{T}(f::Foo)` method and then changing the `convert` method to

```julia
Base.convert(::Type{Foo{T}}, f::Foo) where {T} = Foo(convert(T, f.x))

```

---

<div class="post-metadata">

**Author:** ![arnelg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/arnelg/32/31530_2.png) [@arnelg](https://discourse.julialang.org/u/arnelg)\
**Post date:** [January 10, 2023, 5:19pm UTC](https://discourse.julialang.org/t/how-is-the-type-of-a-heterogeneous-vector-determined-when-splatting-and-doing-list-comprehension/92572/3 "2023-01-10T17:19:46Z")

</div>

Thanks Steven. That indeed leads to running code for all three cases. With that solution,“case 3”,

```julia
[x for x in Any[f1, f2]]

```

gives a `Vector{Foo}` which is the default behavior. I wonder how to get this to become a `Vector{Foo{Real}}` though? Is there some other function than `promote_rule` that has to have a method, something that specifies the first common supertype or something like that? It’s still opaque to me what happens under the hood for “case 3”.

---

<div class="post-metadata">

**Author:** ![StevenWhitaker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevenwhitaker/32/9749_2.png) [@StevenWhitaker](https://discourse.julialang.org/u/StevenWhitaker)\
**Post date:** [January 12, 2023, 5:20pm UTC](https://discourse.julialang.org/t/how-is-the-type-of-a-heterogeneous-vector-determined-when-splatting-and-doing-list-comprehension/92572/4 "2023-01-12T17:20:15Z")

</div>

The reason why the result is a `Vector{Foo}` is because `Foo` _is_ the first common supertype of `Foo{Int64}` and `Foo{Float64}`. Importantly, `Foo{Real}` is a _concrete_ type, and thus

```julia
julia> Foo{Int64} <: Foo{Real}
false

```

so a `Vector{Foo{Real}}` would have to be constructed with objects that are all of type `Foo{Real}`.

---

<div class="post-metadata">

**Author:** ![uniment](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/uniment/32/24532_2.png) [@uniment](https://discourse.julialang.org/u/uniment)\
**Post date:** [January 12, 2023, 11:22pm UTC](https://discourse.julialang.org/t/how-is-the-type-of-a-heterogeneous-vector-determined-when-splatting-and-doing-list-comprehension/92572/5 "2023-01-12T23:22:39Z")

</div>

> [@StevenWhitaker](#):
>
> `Foo` _is_ the first common supertype

Not quite; `Foo{<:Real}` is the first common supertype.

```julia
julia> (Foo{Int}, Foo{Float64}) .<: Foo{T} where T<:Real
(true, true)

julia> Foo{<:Real}[Foo(1), Foo(1.0)]
2-element Vector{Foo{<:Real}}:
 Foo{Int64}(1)
 Foo{Float64}(1.0)

```

---

<div class="post-metadata">

**Author:** ![arnelg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/arnelg/32/31530_2.png) [@arnelg](https://discourse.julialang.org/u/arnelg)\
**Post date:** [January 12, 2023, 11:32pm UTC](https://discourse.julialang.org/t/how-is-the-type-of-a-heterogeneous-vector-determined-when-splatting-and-doing-list-comprehension/92572/6 "2023-01-12T23:32:47Z")

</div>

That `Foo{Int64} <: Foo{Real}` is `false` always trips me up. Would it make sense for `[x for x in Any[f1, f2]]` to return a `Vector{Foo{<:Real}}`, and if so, what method(s) would I need to overload to make that happen?

---

<div class="post-metadata">

**Author:** ![uniment](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/uniment/32/24532_2.png) [@uniment](https://discourse.julialang.org/u/uniment)\
**Post date:** [January 13, 2023, 12:47am UTC](https://discourse.julialang.org/t/how-is-the-type-of-a-heterogeneous-vector-determined-when-splatting-and-doing-list-comprehension/92572/7 "2023-01-13T00:47:14Z")

</div>

This seems to do the trick:

```julia
julia> struct Foo{T}
           x::T
       end

julia> Base.promote_rule(::Type{Foo{T}}, ::Type{Foo{S}}) where {T, S} =
           Foo{promote_type(T, S)}

julia> Base.convert(::Type{Foo{T}}, f::Foo) where {T} = Foo(convert(T, f.x))

julia> Base.typejoin(::Type{Foo{T1}}, ::Type{Foo{T2}}) where {T1,T2} = Foo{T} where {T<:typejoin(T1, T2)}

julia> [x for x ∈ Any[Foo(1), Foo(1.0)]]
2-element Vector{Foo{T} where T<:Real}:
 Foo{Int64}(1)
 Foo{Float64}(1.0)

```

---

<div class="post-metadata">

**Author:** ![StevenWhitaker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevenwhitaker/32/9749_2.png) [@StevenWhitaker](https://discourse.julialang.org/u/StevenWhitaker)\
**Post date:** [January 13, 2023, 5:17pm UTC](https://discourse.julialang.org/t/how-is-the-type-of-a-heterogeneous-vector-determined-when-splatting-and-doing-list-comprehension/92572/8 "2023-01-13T17:17:46Z")

</div>

> [@uniment](#):
>
> Not quite; `Foo{<:Real}` is the first common supertype.

Not quite; `Foo{<:Union{Int,Float64}}` is the first common supertype 😉

The point is, `Foo` is the only explicitly defined type in the tree, and what somebody means by “first common supertype” isn’t necessarily clear. For example, given `Foo{A}` and `Foo{B}`, how would Julia know to take the `Union` of `A` and `B` vs traversing the type trees of `A` and `B` to find a common supertype vs just focusing on the type tree of `Foo`? So I would argue that in some sense `Foo` is the first common supertype (of types explicitly defined in the type tree of `Foo`), and that developers have to tell Julia otherwise if they want a different definition of first common supertype, which seems to be accomplished with `typejoin` (which I didn’t know about until today).

---

<div class="post-metadata">

**Author:** ![uniment](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/uniment/32/24532_2.png) [@uniment](https://discourse.julialang.org/u/uniment)\
**Post date:** [January 13, 2023, 8:21pm UTC](https://discourse.julialang.org/t/how-is-the-type-of-a-heterogeneous-vector-determined-when-splatting-and-doing-list-comprehension/92572/9 "2023-01-13T20:21:29Z")

</div>

> [@StevenWhitaker](#):
>
> `Foo{<:Union{Int,Float64}}` is the first common supertype 😉

I stand corrected. 😅

---

<div class="post-metadata">

**Author:** ![arnelg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/arnelg/32/31530_2.png) [@arnelg](https://discourse.julialang.org/u/arnelg)\
**Post date:** [January 15, 2023, 1:48am UTC](https://discourse.julialang.org/t/how-is-the-type-of-a-heterogeneous-vector-determined-when-splatting-and-doing-list-comprehension/92572/10 "2023-01-15T01:48:09Z")

</div>

I also didn’t know about `typejoin` which solves the problem!

One small issue with the proposed solution though is that `convert(Foo{T}, f::Foo)` isn’t guaranteed to return a value of type `Foo{T}`. E.g. `convert(Foo{Any}, Foo(1))` returns `Foo{Int64}(1)` which is not a `Foo{Any}`. So I suppose the correct thing would be to define `Base.convert(::Type{Foo{T}}, f::Foo) where {T} = Foo{T}(f.x)`