if y isa Int
@invoke foo(x::String, y::Int)::Float64
else
@invoke foo(x::String, y::Float64)::Float64
end
This needs to be recompiled when a new foo method appears in a downstream package. But I didn’t quite get why @StefanKarpinski’s suggestion to add a dynamic call as a fallback wouldn’t work out. This would be:
if y isa Int
@invoke foo(x::String, y::Int)::String
elseif y isa Float64
@invoke foo(x::String, y::Float64)::String
else
@dynamic_call foo(x::String, y::Any)::String
end
This looks like a good idea to me? In particular, we could change the rule to “World-splitting only considers methods that are defined in the call-site’s module or in its imported dependencies”. Because that’s the set of methods that are loaded during precompilation, the precompiled code wouldn’t change (beyond adding the fallback). And all methods of foo that are defined downstream and infer to String would no longer invalidate the above code.
One reason is because you don’t know the return type of the dynamic fallback call, so you can no longer infer the type of foo(x::String, y::Any) if you have that present.
Another might be that, as pointed out by the emcee (Tim Holy?), if the caller isn’t precompiled, the runtime performance characteristics would be dependent on load and execution order and thus unpredictable. That is, unless you enforce your rule,
even for methodinstances that are JIT compiled at runtime, but that seems like it would require that the compiler know all about code loading and dependencies, which I assume is not the case at the moment and sounds like it might not be a desireable/feasible coupling. (Caveat: all this is way above my pay grade; this is just how I parsed that part of the talk/QA.)
The idea would be to optimistically assume that the return type of future methods will be of the same type as the existing methods, hence the ::String annotation at the end:
@dynamic_call foo(x::String, y::Any)::String
so it would infer identically.
In other words,
Current world-splitting assumes that all methods of foo are already known. It is invalidated whenever a new foo method appears.
World-splitting with optimistically-typed fallback assumes that all future methods of foo will return a String. It is invalidated whenever a new foo method does not infer to String.
It’s not a panacea, but it would likely solve the convert(::Type{String}, x) example from @topolarity in practice.
Yeah, it’s a decent idea in the vein of “have the compiler guess / infer the interface” for an “extensible” call.
We’d need a bit of extra compiler machinery to be able to notice that a new registered method doesn’t invalidate the “interface” the compiler decided to infer here, but I think it’s a sensible proposal.
It’s not a panacea, but it would likely solve the convert(::Type{String}, x) example
Yeah, it does ultimately end up in the “heuristic” space. It has a couple of likely failures:
The compiler still has to “guess” at the interface here. It might do that successfully for, e.g., a concrete return type, but anything else is likely to be a wrong guess, which invalidates.
New methods that return ::String but don’t infer as returning ::String (i.e. inference was imprecise) will still cause invalidation.
After the discussions at JuliaCon I was wondering if a simple way to recover optimizations for “final interfaces” would be to allow closing abstract types. It would then not be allowed to add any other subtypes to it, making dispatch on it behave like that on a union of all its current subtypes. That would be an easy way to recover performance in cases where people were relying on world splitting in this case:
abstract type Abs end
struct ConcreteEither <: Abs end
struct ConcreteOr <: Abs end
close_type!(Abs)
f(x::Vector{Abs}) = ... # this could now not be invalidated anymore
I think this is quite clever and probably would help “cure” many invalidations. It’d be interesting to collect data on exactly how many invalidations are of this flavor.
My hunch, though, is that there are probably just as many with a slightly different — and harder to divine — pattern:
if y isa Int
@invoke foo(x::String, y::Int)::Int
elseif y isa Float64
@invoke foo(x::String, y::Float64)::Float64
else
Here, you’d want to extrapolate the dynamic call to:
But this is a case where y’s type is not inferred, right? That’s why we split the world on y’s type. So the ::typeof(y) wouldn’t help anything, I think?
I would go with ::Union{Int, Float64} in that case. And then if that gets invalidated (because the new method infers to Complex64), give up and go straight to a dynamic call.
Assuming that those signatures reflect the 2 candidate methods, wouldn’t there need to be a y isa Float64 check and a 3rd branch throwing a MethodError for any other type of y? This would instead make the last invoke throw a TypeError for an unsuitable input type.
At first glance, I’d have two concerns.
(1) This would introduce a dependence of the compiled code on which methods were present at time of compilation. In that example, we compiled the caller when foo had 2 candidate methods, thus 2 @invoke branches and a @dynamic_call fallback. If I write a 3rd method, I use the same code’s fallback branch. Say we saved the caller and the now 3 foo methods to a file and evaluate them into another session. The compiled code would have 3 @invoke branches instead (don’t know what the limit to candidate methods are, but I think we’re close to it). I don’t know if this is actually the case, but I was under the impression that the intent is to have the same compiled code for the same state of methods no matter how or when we get there; it seems like reflection and profiling is only feasible this way.
(2) We wouldn’t need to invalidate the caller’s code, but we do have to check whether to do that. We’d need to check the callers rather than the foo function because we generally can’t know the candidate methods or their calls’ return types that the caller inferred; one caller’s branch can look very different from another caller’s branch. I don’t know how invalidation is implemented, but can this check be done alongside invalidation without much overhead? Can this check and possible invalidation be done on demand instead of all at once upon defining a method?