`@btime` appears to force specialization, does this give misleading timings?

General Usage
2

This isn’t really method specialization but rather inlining with constant propagation.

That is, there’s not really a f(::Type{Int8}) method instance compiled. What there is, however, is a benchmark kernel with a hardcoded/inlined () specifically for ::Type{Int8}. You also see this behavior with a simple outer function:

julia> f(x::Type) = fieldnames(x)
f (generic function with 1 method)

julia> methodinstances(f)
Core.MethodInstance[]

julia> g() = f(Int8)
g (generic function with 1 method)

julia> g()
()

julia> methodinstances(f)
1-element Array{Core.MethodInstance,1}:
 MethodInstance for f(::Type{Int8})

In other words, that method instance only exists to track the backedge to invalidate g() if need be:

julia> methodinstances(f)[1].backedges
1-element Array{Any,1}:
 MethodInstance for g()

This is behaving exactly as I’d expect — the core mantra of BenchmarkTools is to “benchmark the snippet as though it were written directly in a function.” In this case, were you to hardcode Int8 like that, you’d see exactly this performance.

Now, I initially thought that using $ to properly flag the Int8 as an argument instead of a hardcoded constant literal — that is, @btime f($Int8) — would make this behave as you initially expected. But we see the same behavior. The benchmark kernel must indeed be forcing specialization on its arguments internally… and that really does make sense because it needs to properly pass types to functions that are themselves properly specialized (e.g., @btime round($Int8, 2.3)). I don’t really know of a way around that.

One way to see that this is indeed all stemming from inlining is with @noinline:

# new session
julia> using MethodAnalysis

julia> @noinline f(x::Type) = fieldnames(x)
f (generic function with 1 method)

julia> g() = f(Int8)
g (generic function with 1 method)

julia> methodinstances(f)
Core.MethodInstance[]

julia> g()
()

julia> methodinstances(f)
2-element Array{Core.MethodInstance,1}:
 MethodInstance for f(::Type{Int8})
 MethodInstance for f(::Type{T} where T)

Now we actually create the pessimized f(::Type) specialization. We still do also have a Type{Int8} method instance — and I think that’s because it hardcoded which method of f should be called into g() — and that hardcoding of the dispatch itself may need to be invalidated. Again, with a “ghost” method instance that doesn’t actually contain a function pointer:

julia> methodinstances(f)[1].cache.invoke
Ptr{Nothing} @0x0000000000000000

julia> methodinstances(f)[2].cache.invoke
Ptr{Nothing} @0x0000000109e3b490
7 Likes