Union causes incorrect type parameter extraction?

tomtom · August 6, 2021, 4:34pm

function fun(x::Union{AbstractArray{T}, AbstractArray{Union{T, Missing}}};
            α::T = T(3.0) ) where {T<:AbstractFloat}
    println(typeof(x))
end

julia> fun([1.0, 2.0])
Vector{Float64}

julia> fun([1.0, missing, 2.0])
Vector{Union{Missing, Float64}}

on above, everything is fine with only one input belongs to either AbstractArray{T} or AbstractArray{Union{T, Missing}}.

Now, for a function with two those inputs:

function gun(x::Union{AbstractArray{T}, AbstractArray{Union{T, Missing}}},
            y::Union{AbstractArray{T}, AbstractArray{Union{T, Missing}}};
            α::T = T(3.0) ) where {T<:AbstractFloat}
    println(typeof(x), " ", typeof(y))
end

julia> gun([1.0, 2.0], [3.0, 4.0])
Vector{Float64} Vector{Float64}

julia> gun([1.0, 2.0], [3.0, 4.0, missing])
Vector{Float64} Vector{Union{Missing, Float64}}

julia> gun([1.0, missing, 2.0], [3.0, 4.0, missing])
ERROR: MethodError: no method matching Union{Missing, Float64}(::Float64)
Stacktrace:
 [1] gun(x::Vector{Union{Missing, Float64}}, y::Vector{Union{Missing, Float64}})
   @ Main ./REPL[38]:3
 [2] top-level scope
   @ REPL[43]:1

error seems coming from mistaking Union{Missing, Float64} as T, which should be Float64, in the default keyword argument assignment.

noted that if I supply the keyboard argument such that the default value assignment is not called, no error:

julia> gun([1.0, missing, 2.0], [3.0, 4.0, missing]; α = 1.0)
Vector{Union{Missing, Float64}} Vector{Union{Missing, Float64}}

it seems to be a bug in extracting the T (when Union is involved?). using v1.6.1. thanks.

pdeffebach · August 6, 2021, 5:17pm

I can’t replicate exactly. At least without knowing what AbF is.

julia> function fun_xyz(x::Union{AbstractArray{T}, AbstractArray{<:Union{T, Missing}}},
                        y::Union{AbstractArray{T}, AbstractArray{<:Union{T, Missing}}};
                        α::T = T(3.0) ) where {T<:Real}
           println(typeof(x), " ", typeof(y))
       end
fun_xyz (generic function with 1 method)

julia> fun_xyz([1], [2])
Vector{Int64} Vector{Int64}

julia> fun_xyz([1], [2, missing])
Vector{Int64} Vector{Union{Missing, Int64}}

julia> fun_xyz([1, missing], [2, missing])
Vector{Union{Missing, Int64}} Vector{Union{Missing, Int64}}

tomtom · August 7, 2021, 1:48am

oh sorry, AbF is AbstractFloat in my own abbreviation system:

julia> AbF
AbstractFloat

lhnguyen-vn · August 7, 2021, 6:12am

Determining static parameters with Union is generally quite difficult, but it’s worth submitting a bug report in this case since the algorithm is surprisingly not respecting the type constraint.

In the meantime, however, this works:

julia> foo(x::AbstractArray{<:Union{T,Missing}}, y::AbstractArray{<:Union{T,Missing}}) where {T<:AbstractFloat} = T
foo (generic function with 1 method)

julia> foo([1.], [1.])
Float64

julia> foo([1., missing], [1.])
Float64

julia> foo([1., missing], [1., missing])
Float64

tomtom · August 7, 2021, 12:58pm

it works because it does not involve a default value assignment in the keyword argument, which is where the problem occurs

tomtom · August 7, 2021, 12:59pm

i guess u can replicate now?

genkuroki · August 7, 2021, 1:13pm

I also can’t reproduce it. I’m using Julia v1.6.2. Please upgrade the Julia version.

Code:

function fun(x::Union{AbstractArray{T}, AbstractArray{Union{T, Missing}}};
            α::T = T(3.0) ) where {T<:AbstractFloat}
    println(typeof(x))
end

function gun(x::Union{AbstractArray{T}, AbstractArray{Union{T, Missing}}},
            y::Union{AbstractArray{T}, AbstractArray{Union{T, Missing}}};
            α::T = T(3.0) ) where {T<:AbstractFloat}
    println(typeof(x), " ", typeof(y))
end

@show VERSION
fun([1.0, 2.0])
fun([1.0, missing, 2.0])
gun([1.0, 2.0], [3.0, 4.0])
gun([1.0, 2.0], [3.0, 4.0, missing])
gun([1.0, missing, 2.0], [3.0, 4.0, missing])
gun([1.0, missing, 2.0], [3.0, 4.0, missing]; α = 1.0)

Output (no error):

VERSION = v"1.6.2"
Vector{Float64}
Vector{Union{Missing, Float64}}
Vector{Float64} Vector{Float64}
Vector{Float64} Vector{Union{Missing, Float64}}
Vector{Union{Missing, Float64}} Vector{Union{Missing, Float64}}
Vector{Union{Missing, Float64}} Vector{Union{Missing, Float64}}

tomtom · August 7, 2021, 2:35pm

Strange! the problem persists on my side even updated to v1.6.2 !!!

julia> @show VERSION
VERSION = v"1.6.2"
v"1.6.2"

julia> gun([1.0, missing, 2.0], [3.0, 4.0, missing])
ERROR: MethodError: no method matching Union{Missing, Float64}(::Float64)
Stacktrace:
 [1] gun(x::Vector{Union{Missing, Float64}}, y::Vector{Union{Missing, Float64}})
   @ Main ./REPL[2]:4
 [2] top-level scope
   @ REPL[7]:1

I’m using Mac.

tomtom · August 7, 2021, 2:40pm

could u please nicely teach me how to submit a (detailed enough) bug report? Especially in this case seems like some other guys could not replicate the situation…

genkuroki · August 7, 2021, 2:51pm

Wow! I’m using Windows. What is going on behind it?

tomtom · August 9, 2021, 6:43am

hello. could u please try to replicate?

pdeffebach · August 9, 2021, 12:27pm

Yes, I can replicate the error now. I have no idea what’s happening. Please file an issue with the following code

In a fresh session:

julia> function f1(x::Union{AbstractArray{T}, AbstractArray{Union{T, Missing}}},
                    y::Union{AbstractArray{T}, AbstractArray{Union{T, Missing}}};
                    α::T = T(3.0) ) where {T<:Real}
           println(typeof(x), " ", typeof(y))
       end;

julia> f1([1.0, missing], [2.0, missing])
ERROR: MethodError: no method matching Union{Missing, Float64}(::Float64)
Stacktrace:
 [1] f1(x::Vector{Union{Missing, Float64}}, y::Vector{Union{Missing, Float64}})
   @ Main ./REPL[1]:4
 [2] top-level scope
   @ REPL[2]:1

julia> const AbF = AbstractFloat
AbstractFloat

julia> function f2(x::Union{AbstractArray{T}, AbstractArray{<:Union{T, Missing}}},
                   y::Union{AbstractArray{T}, AbstractArray{<:Union{T, Missing}}};
                   α::T = T(3.0) ) where {T<:AbF}
           println(typeof(x), " ", typeof(y))
       end;

julia> f2([1.0, missing], [2.0, missing])
Vector{Union{Missing, Float64}} Vector{Union{Missing, Float64}}

Also in a fresh session

julia> const AbF = AbstractFloat
AbstractFloat

julia> function f2(x::Union{AbstractArray{T}, AbstractArray{<:Union{T, Missing}}},
                   y::Union{AbstractArray{T}, AbstractArray{<:Union{T, Missing}}};
                   α::T = T(3.0) ) where {T<:AbF}
           println(typeof(x), " ", typeof(y))
       end;

julia> f2([1.0, missing], [2.0, missing])
Vector{Union{Missing, Float64}} Vector{Union{Missing, Float64}}

tomtom · August 9, 2021, 3:00pm

issue submitted here. Thanks.

jameson · August 9, 2021, 4:08pm

It sounds like you are attempting to make this parametric in the wrong place, so your constraints don’t work. I think this version will work:

function fun(x::AbstractArray{T}; α=3.0) where {T<:Union{AbstractFloat,Missing}}
    T = nonmissingtype(T)
    α = T(α)
...

tomtom · August 10, 2021, 3:17am

yes it “works”:

julia> function fun1(x::AbstractArray{T};
                       α=3.0) where {T<:Union{AbstractFloat,Missing}}
           T1 = nonmissingtype(T)
           α = T1(α)
           println(typeof(x))
       end
fun1 (generic function with 1 method)

julia> function fun2(x::AbstractArray{T},
                       y::AbstractArray{T};
                       α=3.0) where {T<:Union{AbstractFloat,Missing}}
           T1 = nonmissingtype(T)
           α = T1(α)
           println(typeof(x), " ", typeof(y))
       end
fun2 (generic function with 1 method)

julia> fun1([2.0, missing])
Vector{Union{Missing, Float64}}

julia> fun2([2.0, missing], [3.0, missing])
Vector{Union{Missing, Float64}} Vector{Union{Missing, Float64}}

but it’s a bit weird:

we need another variable T1, rather than the type parameter, to capture a Float64
the type of α may change, i.e. unstable type

so… seems like it’s a strange solution… but maybe it’s the way to help type inference as the original problem requires some “AI inference” that is very difficult to implement…

Sukera · August 10, 2021, 6:51am

Maybe I’m missing something here, but this is never type unstable. If it’s a Union{T, Missing}, it will always be T. If it’s just a T, it’s always a T. The difference is known at compile time, because the eltype of the vector is known at compile time, thus the compiler knows about it and at least two methods will be compiled, making it type stable.

Topic		Replies	Views
Understanding the use of Union with parametric types General Usage parametric-types	3	894	November 16, 2016
Function with vector of Union-Type General Usage function , value-types , missing-values , union	4	505	October 17, 2022
Subtyping with union as type parameter General Usage	3	391	December 16, 2019
Function signature doesn't match when using Union General Usage	3	574	May 30, 2018
Union vs. <: AbstractType New to Julia	7	704	December 28, 2020

Union causes incorrect type parameter extraction?

Related topics