# Small union type failure

**URL:** <https://discourse.julialang.org/t/small-union-type-failure/18640>\
**Category:** General Usage\
**Created:** [December 13, 2018, 9:30pm UTC](https://discourse.julialang.org/t/small-union-type-failure/18640 "2018-12-13T21:30:27Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![cstjean](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cstjean/32/1444_2.png) [@cstjean](https://discourse.julialang.org/u/cstjean)\
**Post date:** [December 13, 2018, 9:30pm UTC](https://discourse.julialang.org/t/small-union-type-failure/18640/1 "2018-12-13T21:30:27Z")

</div>

I thought that `missing` wasn’t treated specially, and that all small type unions would be `Union`ed. Have I misunderstood?

```julia
julia> struct Blag end

julia> typeof([ifelse(randn() < 0, 1.0, Missing()) for _ in 1:100])
Array{Union{Missing, Float64},1}

julia> typeof([ifelse(randn() < 0, 1.0, Blag()) for _ in 1:100])
Array{Any,1}

```

Why is that not `Union{Blag, Float64}`?

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [December 13, 2018, 11:47pm UTC](https://discourse.julialang.org/t/small-union-type-failure/18640/2 "2018-12-13T23:47:28Z")

</div>

`Missing` is not special in the sense that the representation of `Vector{Union{Blag, Float64}}` is represented efficiently just as `Vector{Union{Missing, Float64}}` is. The promotion machinery treats `Missing` “specially” in that it has this promotion rule:

```julia
promote_rule(::Type{Missing}, ::Type{T}) where {T} = Union{Missing, T}

```

If you define a similar promotion rule for `Blag` then I would have thought it should behave similarly, but for some reason that doesn’t work:

```julia
julia> Base.promote_rule(::Type{Blag}, ::Type{T}) where {T} = Union{Blag, T}

julia> typeof([ifelse(randn() < 0, 1.0, Blag()) for _ in 1:100])
Array{Any,1}

```

Perhaps @nalimilan who (IIRC) implemented this special behavior knows what’s up.

---

<div class="post-metadata">

**Author:** ![tkoolen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkoolen/32/1603_2.png) [@tkoolen](https://discourse.julialang.org/u/tkoolen)\
**Post date:** [December 14, 2018, 2:36am UTC](https://discourse.julialang.org/t/small-union-type-failure/18640/3 "2018-12-14T02:36:51Z")

</div>

Are you sure that promotion is actually involved in this operation?

If anybody’s interested, to see the relevant `code_warntype`, run:

```julia
struct Blag end
genblag = Base.Generator(i -> ifelse(randn() < 0, Blag(), 1.0), 1:100)
@code_warntype collect(genblag)
genmissing = Base.Generator(i -> ifelse(randn() < 0, Missing(), 1.0), 1:100)
@code_warntype collect(genmissing)

```

Not sure why, but the difference appears to be related to `collect_to_with_first!`:

> <https://github.com/JuliaLang/julia/blob/77a7d92e91769146435fe92548d253fa18740840/base/array.jl#L626-L635>

One interesting thing is that some code blocks seem to be reordered. Another is that the inference result for `genblag` is actually tighter than than for `genmissing` (`Array` vs. `AbstractArray`).

---

<div class="post-metadata">

**Author:** ![tkoolen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkoolen/32/1603_2.png) [@tkoolen](https://discourse.julialang.org/u/tkoolen)\
**Post date:** [December 14, 2018, 2:55am UTC](https://discourse.julialang.org/t/small-union-type-failure/18640/4 "2018-12-14T02:55:21Z")

</div>

Ah, found the difference I think:

```julia
julia> Base.promote_typejoin(Float64, Blag)
Any

julia> Base.promote_typejoin(Float64, Missing)
Union{Missing, Float64}

```

That’s being called here:

> <https://github.com/JuliaLang/julia/blob/77a7d92e91769146435fe92548d253fa18740840/base/array.jl#L639>

There are overloads for `Missing`:

> <https://github.com/JuliaLang/julia/blob/77a7d92e91769146435fe92548d253fa18740840/base/promotion.jl#L134-L137>

So after defining

```julia
Base._promote_typejoin(::Type{Blag}, ::Type{T}) where {T} =
    isconcretetype(T) || T === Union{} ? Union{T, Blag} : Any
Base._promote_typejoin(::Type{T}, ::Type{Blag}) where {T} =
    isconcretetype(T) || T === Union{} ? Union{T, Blag} : Any

```

you get

```julia
julia> typeof([ifelse(randn() < 0, 1.0, Blag()) for _ in 1:100])
Array{Union{Blag, Float64},1}

```

---

<div class="post-metadata">

**Author:** ![nalimilan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nalimilan/32/147_2.png) [@nalimilan](https://discourse.julialang.org/u/nalimilan)\
**Post date:** [December 14, 2018, 8:13am UTC](https://discourse.julialang.org/t/small-union-type-failure/18640/5 "2018-12-14T08:13:48Z")

</div>

Yes, currently `Missing` and `Nothing` are special-cased in `promote_typejoin`, because it wasn’t clear how to generalize it: what is a “small” type union? We could pick an arbitrary threshold, but that would give inconsistent behaviors when you add more types.

For now I guess you can overload `promote_typejoin` if you define a special type which needs the same treatment, but beware that it’s unexported and can therefore change at any point. It would be nice to decide whether that mechanism is there to stay, or whether we can find better, more general rules.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [December 14, 2018, 9:17am UTC](https://discourse.julialang.org/t/small-union-type-failure/18640/6 "2018-12-14T09:17:48Z")

</div>

I wonder if something along the lines of

```julia
function Base._promote_typejoin(::Type{S}, ::Type{T}) where {S, T}
    _cs(T, S) = (isconcretetype(T) || T ≡ Union{}) && Base.issingletontype(S)
    if _cs(T, S) || _cs(S, T)
        Union{T, S}
    else
        typejoin(S, T)
    end
end

```

would work, generalizing the existing cases to the union of a concrete and a singleton type.

---

<div class="post-metadata">

**Author:** ![piever](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/piever/32/1815_2.png) [@piever](https://discourse.julialang.org/u/piever)\
**Post date:** [December 14, 2018, 10:01am UTC](https://discourse.julialang.org/t/small-union-type-failure/18640/7 "2018-12-14T10:01:08Z")

</div>

This is probably very basic, but I can’t quite figure it out. Why is `promote_typejoin` necessary given that `promote_type(Int, Missing) = Union{T, Missing}`?

I’m asking because I’m implementing the collection mechanism to collect an iterator of structs into a struct of arrays and have been using `promote_type` and it seemed to work fine for `Missing` as well (code [here](https://github.com/piever/StructArrays.jl/blob/master/src/collect.jl)).

---

<div class="post-metadata">

**Author:** ![nalimilan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nalimilan/32/147_2.png) [@nalimilan](https://discourse.julialang.org/u/nalimilan)\
**Post date:** [December 14, 2018, 5:30pm UTC](https://discourse.julialang.org/t/small-union-type-failure/18640/8 "2018-12-14T17:30:58Z")

</div>

`promote_type` is only used in some places, like `cat`, but other functions like `collect` and `map` don’t use it since preserve the types of elements and therefore merely choose an eltype which is a supertype of all entries’ types. IOW no promotion/conversion happens. `promote_typejoin` is used in that case: it’s really just `typejoin`, which a special case for `Missing` and `Nothing` (the “promote” in the name is a bit misleading since it doesn’t use promotion).

---

<div class="post-metadata">

**Author:** ![piever](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/piever/32/1815_2.png) [@piever](https://discourse.julialang.org/u/piever)\
**Post date:** [December 14, 2018, 6:27pm UTC](https://discourse.julialang.org/t/small-union-type-failure/18640/9 "2018-12-14T18:27:46Z")

</div>

I see, so basically it’s kind of a judgement call in the collection mechanism whether we want to automatically promote (so that mixing `Int` and `Float64` gives `Float64`, like in `vcat`) or whether we take the supertype (so mixing `Int` and `Float64` gives `Real`, like in `collect`), with the option of taking the supertype expect when one of the types is `Missing` or `Nothing` (using `Base.promote_typejoin`). I understand that before this is explicitly documented it’s better to be 100% sure whether this is our definite solution or if there is somehow a simpler solution.

---

<div class="post-metadata">

**Author:** ![cstjean](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cstjean/32/1444_2.png) [@cstjean](https://discourse.julialang.org/u/cstjean)\
**Post date:** [December 31, 2019, 7:00pm UTC](https://discourse.julialang.org/t/small-union-type-failure/18640/10 "2019-12-31T19:00:42Z")

</div>

```julia
julia> [1.0, missing, nothing]
3-element Array{Union{Missing, Nothing, Float64},1}:
 1.0     
  missing
  nothing

julia> identity.([1.0, missing, nothing])
3-element Array{Any,1}:
 1.0     
  missing
  nothing

```

This one is a bummer, too. It seems that broadcasting/comprehensions give up on the type whenever there’s more than two involved.

---

<div class="post-metadata">

**Author:** ![cstjean](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cstjean/32/1444_2.png) [@cstjean](https://discourse.julialang.org/u/cstjean)\
**Post date:** [April 8, 2021, 3:03pm UTC](https://discourse.julialang.org/t/small-union-type-failure/18640/11 "2021-04-08T15:03:01Z")

</div>

> [@nalimilan](#):
>
> For now I guess you can overload `promote_typejoin` if you define a special type which needs the same treatment, but beware that it’s unexported and can therefore change at any point.

The doom bell has rung, [Preserve non-concrete types in promote\_typejoin by vtjnash · Pull Request #37019 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/pull/37019) removed `_promote_typejoin`. Does anyone know what should be overloaded now? `promote_typejoin`?

---

<div class="post-metadata">

**Author:** ![nalimilan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nalimilan/32/147_2.png) [@nalimilan](https://discourse.julialang.org/u/nalimilan)\
**Post date:** [April 8, 2021, 4:16pm UTC](https://discourse.julialang.org/t/small-union-type-failure/18640/12 "2021-04-08T16:16:32Z")

</div>

I guess we need to find a way to allow custom types to opt-in to be included into the list of special types used by `_promote_typesubtract`. Not sure how to do that.

---

<div class="post-metadata">

**Author:** ![cstjean](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cstjean/32/1444_2.png) [@cstjean](https://discourse.julialang.org/u/cstjean)\
**Post date:** [April 8, 2021, 5:12pm UTC](https://discourse.julialang.org/t/small-union-type-failure/18640/13 "2021-04-08T17:12:37Z")

</div>

Ouch, yeah, looks like we need to pirate the base method at the moment?

---

<div class="post-metadata">

**Author:** ![etpinard](https://avatars.discourse-cdn.com/v4/letter/e/45deac/32.png) [@etpinard](https://discourse.julialang.org/u/etpinard)\
**Post date:** [April 13, 2021, 7:43pm UTC](https://discourse.julialang.org/t/small-union-type-failure/18640/14 "2021-04-13T19:43:24Z")

</div>

Hi, using Julia 1.6.0 consider the following

```nohighlight
julia> struct Blag end

# N.B. now `promote_typejoin` not `_promote_typejoin`
julia> Base.promote_typejoin(::Type{Blag}, ::Type{T}) where {T} =
           isconcretetype(T) || T === Union{} ? Union{T, Blag} : Any
julia> Base.promote_typejoin(::Type{T}, ::Type{Blag}) where {T} =
           isconcretetype(T) || T === Union{} ? Union{T, Blag} : Any

# we get the desired outcome
julia> typeof([ifelse(randn() < 0, 1.0, Blag()) for _ in 1:100])
Vector{Union{Blag, Float64}} (alias for Array{Union{Blag, Float64}, 1})

```

So, is it a bad idea to overload `Base.promote_typejoin`?

---

<div class="post-metadata">

**Author:** ![JeffreySarnoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeffreysarnoff/32/1980_2.png) [@JeffreySarnoff](https://discourse.julialang.org/u/JeffreySarnoff)\
**Post date:** [April 15, 2021, 9:23am UTC](https://discourse.julialang.org/t/small-union-type-failure/18640/15 "2021-04-15T09:23:12Z")

</div>

Avoid overloading Base.\<anything\>. with an alternate implementation for the same signature.  
If you feel you have a better way, submit a PR with the revision.

---

<div class="post-metadata">

**Author:** ![foobar\_lv2](https://avatars.discourse-cdn.com/v4/letter/f/ee59a6/32.png) [@foobar\_lv2](https://discourse.julialang.org/u/foobar_lv2)\
**Post date:** [April 15, 2021, 10:52am UTC](https://discourse.julialang.org/t/small-union-type-failure/18640/16 "2021-04-15T10:52:05Z")

</div>

> [@etpinard](#):
>
> So, is it a bad idea to overload `Base.promote_typejoin` ?

In general, overloading Base methods with custom types is very much intended. Type piracy is pretty discouraged, though. (type piracy = overloading methods owned by other modules with types that are not owned by your module)

This specific case is unfortunately bad, because it does not compose:

```julia
julia> struct Blag2 end

julia> Base.promote_typejoin(::Type{Blag2}, ::Type{T}) where {T} =
       isconcretetype(T) || T === Union{} ? Union{T, Blag2} : Any

julia> Base.promote_typejoin(::Type{T}, ::Type{Blag2}) where {T} =
       isconcretetype(T) || T === Union{} ? Union{T, Blag2} : Any

julia> typeof([ifelse(randn() < 0, Blag(), Blag2()) for _ in 1:100])
ERROR: MethodError: promote_typejoin(::Type{Blag2}, ::Type{Blag}) is ambiguous. Candidates:
  promote_typejoin(::Type{T}, ::Type{Blag}) where T in Main at REPL[4]:1
  promote_typejoin(::Type{Blag2}, ::Type{T}) where T in Main at REPL[13]:1

```

This is not really bad type piracy (changing behavior of unrelated code just by loading your module), but it is not pretty either (two independent modules might interact suboptimally – i.e. it is up to people who want to (transitively) import both Blag and Blag2 to resolve the conflict, and this resolution _must_ engage in actual “officially discouraged” type piracy)