Improving speed of runtime dispatch detector

General Usage Performance
61

This seems to be the known challenge with closures (performance of captured variables in closures · Issue #15276 · JuliaLang/julia · GitHub)?

62

Well, there is no refactoring needed unless one wants to use your package. I think the statement from your README file

Code which is type stable should safely compile away the check

needs some caveats. As we have seen, one is the modification of function parameters. This may require rewriting some (type stable) code into a form digestible by DispatchDoctor (unless you find an approach without closures).
Another one that I have just come across is @generated functions. They seem to lead to world age problems. I’m going to open an issue about this shortly. So the whole thing appears to be quite tricky.

EDIT: The GitHub issue is here.

1 Like
63

This is exactly why it states should rather than will. But yeah maybe we should note specific cases where it does not.

Also, the package hasn’t even been registered yet :sweat_smile: though I certainly appreciate the rigorous testing of it

We could also go back to generating a separate function and calling it (@Elrod’s idea), which would avoid Julia’s issues with closures, although there are a variety of tricky things with that. For example how to pass arguments if the user has

f(::Type{T}) where {T} = T

The old approach for this was to basically throw an error so that the user always has a symbol for every argument:

f(t::Type{T}) where {T} = T

I guess we could do that again? Maybe we can also inject that symbol from the macro…

Then the other difficult thing is how to deal with cases like Vararg, Type, or Val, which all mess with Julia specialisation rules. A closure makes this simple because the specialisation of the original function will carry over. But a separate stable_call function would have to create those rules itself. (Which we could try @Elrod’s type wrapping idea for)

We could also generate the contents of stable_call within the function which I guess would avoid this (?) at the expense of a lot of identical generated code.

Oh and I think @generated we should just automatically skip. It would require custom behavior anyway. (It should also automatically skip Base.@pure, MacroTools.@capture, Turing.@model, etc)

1 Like
64

Fixed @matthew3245’s reported issues with Fix closure specialization issue by MilesCranmer · Pull Request #8 · MilesCranmer/DispatchDoctor.jl · GitHub

using DispatchDoctor
@stable function g(n)
    n = Int(n)::Int
    n
end
@test g(1) == 1

which now works, and also

julia> struct SmallBitSet{U}
           mask::U
           global _SmallBitSet(mask::U) where U = new{U}(mask)
       end;

julia> @stable function _push(mask::U, iter) where U
           for n in iter
               mask |= one(U) << (Int(n)-1)
           end
           _SmallBitSet(mask)
       end;


julia> _push(UInt(0), ())
SmallBitSet{UInt64}(0x0000000000000000)

I fixed it by going back to the approach where we generate multiple functions. No more closures! And thus no more issues caused by

The downside is that it now relies on Core.kwcall which is an internal function (and thus DispatchDoctor won’t support older Julia versions – @stable will just be a no-op on those).

1 Like