Performance discrepancy for different ways of piping/composing functions for handling runtime dispatch

General Usage Performance
,
9

Thanks for the explanation!!

I think this is a good mental model, and I think in a sense I was following it by defining the method foo1(::Val{N}) where {N}, since this means that I only want a single method defined for all intances of one type: Val—it’s just that it is not a concrete but an abstract type. Although one could argue “same type” only applies to concrete types, as that’s how instances are directly identified by the dispatch system (except for Type and Function, of course).

However, in the case where the “different type” instances are differentiated by their type parameters, as long as these parameters contain enough type information about themselves useful for when they are treated as input values to a function, we can still have type-stable code:

julia> foo3(::Val{<:Integer}) = 1
foo3 (generic function with 1 method)

julia> foo3(::Val{<:AbstractFloat}) = 2
foo3 (generic function with 2 methods)

julia> typePool = (Float16, Float32, Float64, Int8, Int16, Int32, Int64)
(Float16, Float32, Float64, Int8, Int16, Int32, Int64)

julia> b = Val.(getindex.(Ref(typePool), rand(1:7, 1000)));

julia> @code_warntype foo3.(b)
MethodInstance for (::var"##dotfunction#231#2")(::Vector{Val})
  from (::var"##dotfunction#231#2")(x1) @ Main none:0
Arguments
  #self#::Core.Const(var"##dotfunction#231#2"())
  x1::Vector{Val}
Body::Vector{Int64}
1 ─ %1 = Base.broadcasted(Main.foo3, x1)::Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{1}, Nothing, typeof(foo3), Tuple{Vector{Val}}}
│   %2 = Base.materialize(%1)::Vector{Int64}
└──      return %2

On the contrary, for non-Type parameters like N that do not (and currently cannot) carry any type information about themselves, whenever a function directly manipulates these parameters, the code becomes type unstable:

julia> @code_warntype foo1.(a)
MethodInstance for (::var"##dotfunction#240#1")(::Vector{Val})
  from (::var"##dotfunction#240#1")(x1) @ Main none:0
Arguments
  #self#::Core.Const(var"##dotfunction#240#1"())
  x1::Vector{Val}
Body::AbstractVector
1 ─ %1 = Base.broadcasted(Main.foo1, x1)::Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{1}, Nothing, typeof(foo1), Tuple{Vector{Val}}}
│   %2 = Base.materialize(%1)::AbstractVector
└──      return %2

This is why I think not having complete type annotation capability for the signature of parametric types can also lead to avoidable performance overhead.