Performance changing dramatically due to function redefinition—that smells like problem with the unreliable approximation of `Core.Compiler.return_type` · Issue #35800 · JuliaLang/julia · GitHub to me. See the following recent thread for another example exposing this: Type instability in nested quadgk calls.
I haven’t run your code yet, will hopefully get around to that later. In any case, you should note that @specialize(arg) in a function signature doesn’t do what you think it does, in fact it doesn’t do anything at all (see Essentials · The Julia Language for details). The way to force specialization is to add a type parameter in the function signature, like function foo(x::T, y) where {T}.