Pretty sure this is a severe compiler bug breaking typeof via just a simple parametrized type

The following short code yields an incorrect false (in both v1.12.6/7) from a typeof call, or respectively, typeof(x) changes during runtime even though x does not.

Short version

struct PT{X}
    field::X

    PT(x::X) where X = new{X}(x)
    PT(x::Array{T,N}) where {T,N} = new{Array{T}}(x)
end

function foo(input)
    type = typeof(input.field)::DataType
    output = PT(type)
    println(PT{DataType} === typeof(output)) # false
    return output
end
output = foo(PT(zeros(2)))
println(PT{DataType} === typeof(output)) # true (in REPL)

Some more details
As far as I can see, the bug is due to false type prediction. When returned to the REPL, the type is again correctly determined.

struct ParaType{X}
    field::X

    ParaType(x::X) where X = new{X}(x)
    # ParaType(x::DataType) = new{DataType}(x) # this line at least yields correct prints...
    # ParaType(::Type{X}) where X = new{DataType}(X) # ...so does this line
    ParaType(x::Array{T,N}) where {T,N} = new{Array{T}}(x) # ...so does removing this line
end

bar(x) = 0
bar(x::ParaType{DataType}) = 1
function foo()
    p = ParaType(zeros(2))
    p = [p][1] # ...so does removing this line

    type = typeof(p.field)::DataType
    println(type) # Vector{Float64}
    output = ParaType(type) # adding ::ParaType{DataType} would lead to the verbatim ERROR: TypeError: in typeassert, expected ParaType{DataType}, got a value of type ParaType{DataType}
    println(ParaType{DataType} === typeof(output)) # false
    println(ParaType{DataType} == typeof(output)) # true

    println(ParaType{DataType} === typeof([output][1])) # false
    println(ParaType{DataType} === typeof([output, output][1])) # true

    println(sum(map(bar, (output, output)))) # 0
    println(sum(map(bar, [output]))) # 1
    println(sum(map(bar, [output, output]))) # 2
end
foo()

I’m not sure it’s meaningful to compare types with === like that. They are == and <: and >:… but I suppose it may be at the root of what you’re seeing in foo(). I know some of this has recently changed with the introduction of Core.AnyType, and while nightly doesn’t fix this I do suspect it may be related.

In any case, you can work around it with a method like:

bar(x::ParaType{<: DataType}) = 1

which is a little funny, because DataType reports it isconcretetype.

Working with dispatch on a particular abstract type parameter is tricky… and especially so with types of types. You may want to use the specialized Type{X} parameter when you construct this thing… but that’ll probably only further tie you in knots. Better if you can avoid it, in my experience.

It’s also interesting to point out if you let foo just return PT{DataType} === typeof(output) that this statically resolves to false: in @code_llvm ... you get ret i8 0.

Related, though I’m not sure how much we should trust it, @code_warntype shows (if I interpret it correctly) that the false comes from comparing PT{X} where X<:(Type{Array{Float64, N}} where N) to PT{DataType}.

I’m not sure if it is a proper bug because comparison with == yields true in the short example and it is probably ill-defined what === means for type comparison.

Just adding a small comment. Please note that I just used === to demonstrate the behaviour in a minimal working example.

I originally found this because julia basically denied A <: Union{A,B} for certain two types A and B, where the lefthand A originated from a similar, though much more complicated context, and the righthand union resulted from an eltype call. This lead to a failure of setindex! as it insisted to convert no matter what I changed. I even ended up with the erratic error message expected A but found A then.

Manual call of invoke(<:, ...) then seemed to put julia in a neverending computation, though after cancelling that call, it then decided the identical call A <: Union{A,B} to indeed become true.

I think whether one should do this is besides the point, though I do not consider it that unsual.

Yes, it’s worth opening an issue. But it’s also worth avoiding parametric T{DataType} types… even if they weren’t buggy, they’re quite unhelpful.

This is the inferred abstract type of output = PT(type), and it just seems wrong. output’s runtime type PT{DataType} in the foo(PT(zeros(2))) call is not a subtype because X can only be Type{Array{Float64,1}}, Type{Array{Float64,2}}, etc, not DataType. Every preceding code_warntype line seems fine, and potentially inferring the same narrower type for type and output.field is good, but output::PT{DataType} is the only correct option. FWIW, I tossed the short version into some old Julia installs (all latest patches), and 1.6-9 says true in both scopes and infers output::PT instead.

== gets it right because it got executed at runtime, while === got resolved statically and is thus vulnerable to incorrect type inference, as eldee pointed out.

In a trimmed version that reproduces this bit of inference and compilation, it is apparent that every fo call retrieves the same x1 at runtime and ought to give the same answer, but forcing inference of an iterated Type makes a difference.

julia> begin # replace first PT constructor with: RefValue(x::T) where {T} = RefValue{T}(x)
         fo(reft::Base.RefValue{<:Type}) = typeof(Base.RefValue(reft[])) === Base.RefValue{DataType}
         x1 = Array{Float64,1}
         typex = Type{Array{Float64,N}} where N
         [  typeof(Base.RefValue(x1))
          fo(Base.RefValue{DataType}(x1)) # same as fo(Base.RefValue(x1))
            only(Base.return_types(Base.RefValue, (Type{x1},)))
          fo(Base.RefValue{Type{x1}}(x1))
            only(Base.return_types(Base.RefValue, (typex,)))
          fo(Base.RefValue{typex}(x1))
          # inline body of last call
          typeof(Base.RefValue(Base.RefValue{typex}(x1)[])) === Base.RefValue{DataType}
         ]
       end
7-element Vector{Any}:
      Base.RefValue{DataType}
  true
      Base.RefValue{DataType}
  true
      Base.RefValue{T} where T<:(Type{Array{Float64, N}} where N)
 false
  true

I like the Ref version. It feels both shorter and more natural. I think every constructor that follows Type(x::X) where X = new{X}(x) is currently broken when called via Type(typeof(x)).

The erratic error convert message I described can for instance be observed via:

function foo()
    rx = Ref(typeof(1))

    y = Number[1][1]
    ry = Ref(typeof(y))

    [rx][1] = ry
    # ERROR: MethodError: Cannot `convert` an object of type 
    #   Base.RefValue{DataType} to an object of type 
    #   Base.RefValue{DataType} 
    # (followed by more details...)
end
foo()

So I guess it is indeed generally best to stay away from this. This and the prior example basically boils down to
typeintersect(Base.RefValue{Type{Int}}, Base.RefValue{DataType}) being empty as opposed to typeintersect(Base.RefValue{Type{Int}}, Base.RefValue{<:DataType}), and the compiler forgetting that indeed DataType can have nontrivial subtypes.

While there are other cases (say trying to manually check the types within a union vector) that do not break anything, I wonder if the performance is tied to aboves actual bug. See for instance:

function test()
    n = 1000
    f = x -> map(typeof, x)
    v = fill(Union{Float64, Int}[1, 1], n)
    @time map(f, v) # 0.002802 seconds (13.01 k allocations: 461.383 KiB)
    println(typeof(map(f, v))) # Vector{Vector{DataType}}

    v = fill([1, 1], n)
    @time map(f, v) # 0.000013 seconds (2.00 k allocations: 86.008 KiB)

    v = fill(Any[1, 1], n)
    @time map(f, v) # 0.000013 seconds (2.00 k allocations: 86.008 KiB)
    return nothing
end
test()

I understand that in many cases, typeof particular behaviour throughout compilation is important. Yet while the output of map is just the same in all cases, it is ironically 100 times slower for the Union than the Any case, since at some point, the compiler tries to infer the possible return types of typeof more specifically than just DataType, even though the true return type of the map call Vector{DataType} is itself type-stable.

Unfortunately, after map, this information is not even available to the compiler, but the compiler can not predict that all subfunctions in map are just called for that very result:

function _test()
    n = 1000
    f = x -> map(typeof, x)
    v = fill(Union{Float64, Int}[1, 1], n)
    m1 = map(f, v)

    v = fill([1, 1], n)
    m2 = map(f, v)

    v = fill(Any[1, 1], n)
    m3 = map(f, v)
    return nothing
end
@code_warntype _test()
# (...)
# m3::Vector{Vector{DataType}}
# m2::Vector{Vector{DataType}}
# m1::Vector
# (...)

One can quickly fix this by first initiating Vector{DataType}(undef, n). Funny enough, one can also wrap the type into a tuple via f = x -> map(e->tuple(typeof(e)),x), which lets all versions run equally fast as well.

So again, this particular case has an easy fix, but it kind of wants me to see a very bright warning in the documentation of typeof or within the performance tips section. The biggest problem I see at the moment is the difficulty to predict the performance of typeof in different situation, in particular in combination with tuples:

function test()
    v = Union{Float64, Int}[1, 1]
    t = (typeof(v[1]),)
    val1 = Val{t[1] <: Number}()
    val2 = Val{typeof(v[1]) <: Number}()
end
@code_warntype test()
# val2::Val{true}
# val1::Val
# t::Tuple{Type}

So julia can not manage to carry over the narrowed types through the Tuple, which leads right into the much discussed rabbit hole that Tuple{Int} <: (Tuple{Type{T}} where T <: Union{Int,Float64}) curiously is false (which it really should not be, but I think this is for some very deep implementation reasons…?).

So there is currently no chance for julia to narrow down the type of t using some Type{...}, but I do not know why julia should currently fail to infer that at least t::Tuple{DataType}.

Unfortunately, some behaviour and documentation within julia can lead to some wrong assumptions on the behaviour of DataType as well:

isconcretetype(DataType) # true
subtypes(DataType) # Type[] (although what follows is...)
Type{Int} <: DataType # true
(Type{T} where T) <: DataType # false (yet Type{T} <: DataType for every T)
subtype(Type{Int}) # 1-element Vector{Any}: Type{Int} (in turn returns the type itself)
subtype(Int) # Type[] (in turn does not return Int)
Base.return_types(typeof) # 1-element Vector{Any}: DataType
<:(T1, T2)::Bool

  Subtyping relation, defined between two types. In Julia, a type S is said to be a subtype of a type T if and only if we have S <: T.
  subtypes(T::DataType)

  Return a list of immediate subtypes of DataType T. Note that all currently loaded subtypes are included, including those not visible in the current module.

This is a known, long-standing bug. Most of the mechanics for the fix is already landed for it in v1.14 with the introduction of Core.AnyType as mbauman mentioned, though some details may still remain to make sure the fix is actually visible here as a fix.

Appears to be caused by a similar static mis-inference of ry, looks funnier because it retrieves the correct type at runtime for the error message.

I don’t see a misinference bug, just less precise yet correct inference putting more work into runtime. It also happens to the map(typeof, x) call inside f, ie map(typeof, Union{Float64, Int64}[1,1]) is inferred as Vector instead of Vector{DataType}, and the map implementation itself likely plays a role because broadcast gets a bit closer with Union{Vector{Type}, Vector{DataType}} and just manages to eliminate the performance discrepancy in the wider test() via union-splitting. In general however, inference in mainstream type systems is undecidable and must be cut short somewhere. Statically typed languages are forced to limit where inference works to guarantee static type information even during dynamic dispatch, but dynamically typed languages are much freer to let types be handled at runtime, at least until we add optional static requirements e.g. JuliaC. I do agree this aspect of dynamic typing should be documented more plainly, but I don’t know where because it’s not particular to typeof and doesn’t seem to yield any performance tips.

This bit is actually proper invariance of non-Tuples ie TI <: DT does not imply R{TI} <: R{DT}.

I think there’s a typo here, it’s not strange for a tuple of integers to be unrelated to a tuple of types. I’d guess Tuple{Type{Int}} <: (Tuple{Type{T}} where T <: Union{Int,Float64}) in context, but that returns true.

subtypes isn’t an exhaustive list of all possible and potentially infinite subtypes, and it’s documented as such. I’d only use it to summarize custom subtypes, not for type system reflection.

Counterexample T::Union or T::UnionAll e.g. Type{Vector}

Not sure what you’re commenting here, but it seems normal. typeof returns a concrete type of the input instance, which itself may be a type, and the type of the returned types is DataType.

The behaviour of map is somewhat particular to typeof, since as far as I can see, it is the only function which output (for DataType inputs) can have a more closely inferred type during the compile time than what typeof will show during runtime.

It would be interesting to see at which exact point when calling map(typeof,v), down into subfunctions and up again, the performance bottleneck appears. So when does julia give up on closer inference but does not conclude that it could stick to Vector{DataType} in the first place.

Concerning the Ref example, I did mean to say that something during compilation forgets that this proper invariance is indeed not compatible with the after all useful type inference ry::Base.RefValue{T} where T<:(Type{<:Number}). Maybe because at some point in compilation, DataType is not properly checked for its subtypes, which it has despite isconcrete(DataType) being true (just as the subtypes function does return Type[]).

I did make a typo with Tuple{Int}. I meant to question that (Int,) isa Tuple{Type{Int}} (quite famously) evaluates to false, just as the compiler does not infer typeof((Int,))::Tuple{Type{Int}} during compilation. Viewing the types system set-theoretically, this seems just wrong. I have seen the argument that this avoids too much specialization, but there are already other mechanics in julia that can avoid specialization (just as foo(x...) is not specialized). It would be nice to be able to use declarations like bar(x::Tuple{Type{Int}}) and thus specialization when desired. I am not sure though whether this does currently cause any bottlenecks in basic routines of julia.

You are also correct with that I skipped details in (Type{T} where T) <: DataType. Again, set-theoretically (and I know this is not all that matters), should not (Type{T} where T <: DataType) <: DataType be true?

I disagree however with that subtypes is documented as such. The texts provided through help on subtypes and <: are short and quite clear, but technically not compatible. Whether it mentions not exhaustive somewhere else is not really the point I would argue.

The thing about Base.return_types(typeof) really comes down to where and how it is used. If used as such during compilation, as I mentioned in the beginning, it would understate that the output of typeof(Int) is first captured as Type{Int}.

quick edit: I noticed that Base.return_types(typeof, Tuple{Int}) indeed gives Type{Int}, though I find that suprising in a different way. Calling typeof(Int) gives one a DataType, which in turn is not subset of Type{Int}. Does this not somehow violate return_types? Maybe I am getting a bit lost here…

I still have the hunch that there may be a relation of all that and the sometimes unreasonably slow allequal, Generators and broadcasting, but I really can not tell.

A type hierarchy with concrete leaf types like DataType fully describes all their instances for a working language implementation. More types to categorize instances in other ways are created for practical purposes, not theoretical, and they need not be public e.g. Core.Const, Core.PartialStruct. I’ve actually asked about (Int,) isa Tuple{Type{Int}} being false on this forum before, my main question being how related calls can dispatch to Type selector-annotated methods. Didn’t get an answer then, but turns out the processing of a call does not involve matching the tuple of inputs to method signature tuple types with isa, so there was no need to make isa work for (abstract) subtypes of concrete tuple types. Counterintuitive, but types are not sets. Languages do not even agree on having a top type like Any, in contrast to our agreement that a universal set cannot exist, so rest assured, type systems being less conventional than ZFC does not limit compiler optimizations.

First, that input is for calls typeof(::Int), not typeof(Int). That obviously returns Int, which has the concrete return type DataType. Type{Int} is reported instead to preserve information about the sole return value. Second, you intended to infer the call typeof(Int) returning DataType, and Base.return_types(typeof, Tuple{Type{Int}}) indeed reports Type{DataType}.

False for T = Union{}, the bottom type. To break this down, the where statement was attempting to cover the unusual abstract subtypes T = Type{X} of DataType, and the outer statement Type{Type{X}} <: DataType is essentially asserting Type{X} isa DataType. Reasonable because any parametric type with all its parameters specified is an instance of DataType. Taking the bottom type into account however, Union{} isa DataType is plainly false because typeof(Union{}) is the concrete Core.TypeofBottom.

Yeah, you are absolutely right about Base.return_types. I originally had a missunterstanding of what its output is without second argument. I was also not aware of how julia treats Union{}. Thanks for pointing that out. I was too invested to maybe find systemic reasons for the initial bug.

There is still some left though, where I can not wrap my head around:

function test()
    w = Number[1]
    x = w[1]
    T = typeof(x)::DataType # does not change anything?
    
    t = (T,)
    v = [T,T]

    t2 = (T,)::Tuple{DataType}
    T2 = t2[1]
    v2 = [T2,T2]
end
@code_warntype test()

Locals
  v2::Vector{DataType}
  T2::DataType
  t2::Tuple{DataType}
  v::Any
  t::Tuple{Type}
  T::Type{<:Number}
  x::Number
  w::Vector{Number}

Julia fails to infer t::Tuple{DataType} or v::Vector{DataType}. It starts with T::Type{<:Number}, which ironically is both sharper and not sharper than DataType, depending on context. Indeed, subsequently, it could theoretically T equal Union{Int,Float64} as that fulfills isa Type{<:Number}, but not isa DataType. So for (T,), it retreats to Tuple{Type}.
But I do not get why explicitly adding ::DataType does not change anything. It only succeeds after the Tuple construction with t2, v2, where you have to somewhat cheat with the Tuple wrapping (which now deliberatley uses that julia will never infer ::Tuple{Type{...}}).

Again, w = Any[1] will ironically lead to v::Vector{DataType}.

Would be nice to know if v1.14 is already working on this in one way or another. In theory, julia could infer both T::Type{<:Number} and T::DataType and then stick to what turns out more useful.

About the set interpretation:

I know that types are not implemented as sets (whatever that would mean), but function dispatch on types can already very nicely be (mostly) broken down to very few rules, where isa and <: can be interpreted as \in and \subseteq, Union as \cup, and in particular Type{T} as singelton set {T}.

However, the system is not defined from bottom to top, but constructed from top to bottom from the universal set Any and declarations through <:. So I would not compare it to ZFC, as far as I can tell.

Still, for a struct T{X}, what so far seems to work for me is to think of the type T{X} as \{ T_{X}(x) \mid x \in X \}, as well as Tuple{X} as \{ Tuple(x) \mid x \in X \}. So one does not have T{X} <: T{Y} just because X <: Y, but Tuple{X} <: Tuple{Y}.

One can then look at Tuple{Type{Y}}. By that interpretation, it can (or rather would) be interpreted as \{ Tuple(x) \mid x \in \{ Y \}\} = \{ Tuple(Y) \}. Since that element in julia equates to (Y,), one would expect (Y,) isa Tuple{Type{Y}}.

The compiler may not actively use that. For instance, @code_warntype shows typeof(1)::Type{Int} but not typeof(typeof(1))::Type{Type{Int}} because typeof(Int) simplifies to DataType, even though it could purely set-theoretically return Type{Int}. But julia does not set Type{Int} isa Type{Type{Int}} as false itself. I know that typeof((Int,)) gives Tuple{DataType} and that this type does not fulfil <: Tuple{Type{Int}}, but that is directly relevant. To evaluate Int isa Type{Int}, julia also does not ask typeof(Int) <: Type{Int}. So there is more to isa than just typeof() <: anyways.

I do not mean though to say that the interpretation above is the only way to look at it, and I know there is more to the actual julia implementation than theoretical, mathematical ideals, but I think it is important to distinguish between necessities (or workarounds) and such ideals. I do not know if there is agreement that julia should (eventually) warn about function foo(x::Tuple{Type{Int}}), because as far as I can see that method is impossible to call, or if (Int,) isa Tuple{Type{Int}} should become true. Maybe you know more.

In theory, 1 literal’s type could be preserved in x::Int, T::Type{Int}, etc, and it wouldn’t require t::Tuple{Type{Int}} to work. But yes, ideally a DataType wouldn’t be discarded only to end up with a less specific Type. At least that example doesn’t appear to overly narrow types and compile incorrect code.

:: assertions aren’t static declarations. Type inference can try to infer the right-hand object as an upper type bound throughout the non-erroring branch, but that doesn’t amount to us inserting type information to give inference a break. Type{<:Number} was narrower than the DataType you provided, so inference chose to stick with that, despite it working out for the worse at t and v.

No idea. It’d be intuitive and convenient, and any tuple type annotation appears to me to have equal information to separate annotated arguments, which already work. But plain Type selectors are what was given to us to distinguish methods by more than ::DataType, nothing else was promised. Making that isa statement true alone won’t make the dispatch would work either; the Type selector dispatch works because Julia special-cased type inputs to be dispatched by Type{X} instead of the concrete type DataType. That doesn’t currently happen for (Int,), and foo(::Tuple{DataType}) doesn’t come close to matching foo(x::Tuple{Type{Int}}).

Possibly these very old conversations might clarify Tuple{Type}?

Maybe I am missinterpreting what you mean, but if you mean that Type{<:Number} is narrower than DataType, this is not true, and neither vice versa.

types = (Int, Union{Int, Float64}, Symbol)
 println.(map(T -> (T, T isa Type{<:Number}, T isa DataType, typeof(T)), types));
# (Int64, true, true, DataType)
# (Union{Float64, Int64}, true, false, Union)
# (Symbol, false, true, DataType)

I have never had a case where manual asserting x::T did not result in the compiler fully assuming that afterwards.

Here, it actually seems hard to bring the compiler to really assume ::DataType. See this slightly absurd code and performance:

struct TypeOf
    value::DataType
    TypeOf(@nospecialize x) = new(typeof(x))
end
_typeof(x)::DataType = TypeOf(x).value
function test()
    x = repeat(Union{Int,Float64}[1,2],10)
    @time for i = 1:1000; map(typeof,x); end
    @time for i = 1:1000; map(_typeof,x); end

    x = repeat(Number[1,2],10)
    @time for i = 1:1000; map(typeof,x); end
    @time for i = 1:1000; map(_typeof,x); end

    x = repeat(Int[1,2],10)
    @time for i = 1:1000; map(typeof,x); end
    @time for i = 1:1000; map(_typeof,x); end
return
end
test()
#   0.005018 seconds (11.01 k allocations: 484.672 KiB)
#   0.004363 seconds (9.00 k allocations: 359.375 KiB)
#   0.005046 seconds (11.00 k allocations: 484.375 KiB)
#   0.000115 seconds (2.00 k allocations: 218.750 KiB)
#   0.000045 seconds (1000 allocations: 187.500 KiB)
#   0.000042 seconds (1000 allocations: 187.500 KiB)

It still does not work for the Union case, but at least for Number.

Thing about (Int,) is that even if it would not work for dispatch, one could still manually do so during runtime via if t isa S when S = Tuple{Type{Int}}. I know one could just use ask if t[1] isa Type{Int} or if t[1] === Int, but when t and S are somewhat generated via a more complicate machinery, any such inconsistency is eventually going to give an unexpected or undesired result without necessarily being noticed. In turn, using foo(::Tuple{Type{Int}}) could just give a warning because catching that during method creating does not really create relevant overhead.

Unfortunately, every discussion I have seen about that case never really seems to be concluded. Can not really find that in the two contributions linked either. I am also not sure which parts of it are still valid since much changed since then.

I didn’t say “narrow” to mean that they were true subtypes, rather what runtime types from typeof are considered during type inference, sorry for the confusing informal term. However, this does make me wonder if the unnecessary inclusion of Union types and maybe others is forcing the fallback to Type instead of DataType.

In most cases where the leaf type is the concrete type and you know you can assert narrower types, this happens reliably. The compiler is still free to not use T e.g. x::rand((Number, Real)) = 1 can thankfully be inferred as x::Int, and the unusual Type{X} <: DataType makes it much more complicated. Compilers of statically typed languages can ignore explicit static types in favor of inferred static information, so a compiler for a dynamically typed language can ignore mere assertions.

That just goes to show how much more opinionated real type systems are than more formalized type theory or set theory. The first link shows that a core language developer just decided against making isa work for an abstract tuple type in the v0 era, and the second link reconsiders the metatypes entirely and acknowledges it requires a breaking v2 that isn’t being seriously considered. Type{Int} <: DataType isn’t even correct for some reason I do not know, and throwing out DataType could potentially make this entire conversation about inferring DataType as a parameter value moot.