Pretty sure this is a severe compiler bug breaking typeof via just a simple parametrized type

Internals & Design
, ,
1

The following short code yields an incorrect false (in both v1.12.6/7) from a typeof call, or respectively, typeof(x) changes during runtime even though x does not.

Short version

struct PT{X}
    field::X

    PT(x::X) where X = new{X}(x)
    PT(x::Array{T,N}) where {T,N} = new{Array{T}}(x)
end

function foo(input)
    type = typeof(input.field)::DataType
    output = PT(type)
    println(PT{DataType} === typeof(output)) # false
    return output
end
output = foo(PT(zeros(2)))
println(PT{DataType} === typeof(output)) # true (in REPL)

Some more details
As far as I can see, the bug is due to false type prediction. When returned to the REPL, the type is again correctly determined.

struct ParaType{X}
    field::X

    ParaType(x::X) where X = new{X}(x)
    # ParaType(x::DataType) = new{DataType}(x) # this line at least yields correct prints...
    # ParaType(::Type{X}) where X = new{DataType}(X) # ...so does this line
    ParaType(x::Array{T,N}) where {T,N} = new{Array{T}}(x) # ...so does removing this line
end

bar(x) = 0
bar(x::ParaType{DataType}) = 1
function foo()
    p = ParaType(zeros(2))
    p = [p][1] # ...so does removing this line

    type = typeof(p.field)::DataType
    println(type) # Vector{Float64}
    output = ParaType(type) # adding ::ParaType{DataType} would lead to the verbatim ERROR: TypeError: in typeassert, expected ParaType{DataType}, got a value of type ParaType{DataType}
    println(ParaType{DataType} === typeof(output)) # false
    println(ParaType{DataType} == typeof(output)) # true

    println(ParaType{DataType} === typeof([output][1])) # false
    println(ParaType{DataType} === typeof([output, output][1])) # true

    println(sum(map(bar, (output, output)))) # 0
    println(sum(map(bar, [output]))) # 1
    println(sum(map(bar, [output, output]))) # 2
end
foo()
2

I’m not sure it’s meaningful to compare types with === like that. They are == and <: and >:… but I suppose it may be at the root of what you’re seeing in foo(). I know some of this has recently changed with the introduction of Core.AnyType, and while nightly doesn’t fix this I do suspect it may be related.

In any case, you can work around it with a method like:

bar(x::ParaType{<: DataType}) = 1

which is a little funny, because DataType reports it isconcretetype.

Working with dispatch on a particular abstract type parameter is tricky… and especially so with types of types. You may want to use the specialized Type{X} parameter when you construct this thing… but that’ll probably only further tie you in knots. Better if you can avoid it, in my experience.

3

It’s also interesting to point out if you let foo just return PT{DataType} === typeof(output) that this statically resolves to false: in @code_llvm ... you get ret i8 0.

Related, though I’m not sure how much we should trust it, @code_warntype shows (if I interpret it correctly) that the false comes from comparing PT{X} where X<:(Type{Array{Float64, N}} where N) to PT{DataType}.

4

I’m not sure if it is a proper bug because comparison with == yields true in the short example and it is probably ill-defined what === means for type comparison.

5

Just adding a small comment. Please note that I just used === to demonstrate the behaviour in a minimal working example.

I originally found this because julia basically denied A <: Union{A,B} for certain two types A and B, where the lefthand A originated from a similar, though much more complicated context, and the righthand union resulted from an eltype call. This lead to a failure of setindex! as it insisted to convert no matter what I changed. I even ended up with the erratic error message expected A but found A then.

Manual call of invoke(<:, ...) then seemed to put julia in a neverending computation, though after cancelling that call, it then decided the identical call A <: Union{A,B} to indeed become true.

I think whether one should do this is besides the point, though I do not consider it that unsual.

6

Yes, it’s worth opening an issue. But it’s also worth avoiding parametric T{DataType} types… even if they weren’t buggy, they’re quite unhelpful.

7

This is the inferred abstract type of output = PT(type), and it just seems wrong. output’s runtime type PT{DataType} in the foo(PT(zeros(2))) call is not a subtype because X can only be Type{Array{Float64,1}}, Type{Array{Float64,2}}, etc, not DataType. Every line before that point seems fine:

  input::PT{Array{Float64}}
Locals
  output::PT{X} where X<:(Type{Array{Float64, N}} where N)
  type::Type{Array{Float64, N}} where N
Body::PT{X} where X<:(Type{Array{Float64, N}} where N)
1 ─ %1  = Main.typeof::Core.Const(typeof)
│   %2  = Base.getproperty(input, :field)::Array{Float64}
│   %3  = (%1)(%2)::Type{Array{Float64, N}} where N
│   %4  = Main.DataType::Core.Const(DataType)
│         (type = Core.typeassert(%3, %4))
│   %6  = Main.PT::Core.Const(PT)
│   %7  = type::Type{Array{Float64, N}} where N
│         (output = (%6)(%7))

FWIW, I tossed the short version into some old Julia installs (all latest patches), and 1.6-9 says true in both scopes and infers output::PT instead.