# Improving speed of runtime dispatch detector

**URL:** <https://discourse.julialang.org/t/improving-speed-of-runtime-dispatch-detector/114697>\
**Category:** Performance\
**Tags:** question\
**Created:** [May 24, 2024, 6:45pm UTC](https://discourse.julialang.org/t/improving-speed-of-runtime-dispatch-detector/114697 "2024-05-24T18:45:59Z")\
**Posts on this page:** 20\
**Page:** 2

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [May 25, 2024, 4:50pm UTC](https://discourse.julialang.org/t/improving-speed-of-runtime-dispatch-detector/114697/21 "2024-05-25T16:50:07Z")

</div>

I don’t know if module-wise is impossible (not saying it isn’t), but could at calling site work?

`@inline` used to only work at the definition, and then later also at the caller site. I’m not sure how it works, or why, I think it’s still one (improved) macro… [I suppose it’s really two in one, and it needs to detect where it’s applied.]

`@inline` is not recursive (nor would you want it to be), but since it’s possible for the calling site, could you do the same, and even recursive…? I mean theoretically, not pushing you do it personally. And I mean `@stable f(x)` though also intriguing would be: `stable using <module>`, could such work?

---

<div class="post-metadata">

**Author:** ![MilesCranmer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/milescranmer/32/21070_2.png) [@MilesCranmer](https://discourse.julialang.org/u/MilesCranmer)\
**Post date:** [May 25, 2024, 5:45pm UTC](https://discourse.julialang.org/t/improving-speed-of-runtime-dispatch-detector/114697/22 "2024-05-25T17:45:34Z")

</div>

Okay all is good, I fixed it [here](https://github.com/MilesCranmer/DispatchDoctor.jl/pull/3) by moving back to the closure approach:

```julia
function _stable(fex::Expr)
    func = splitdef(fex)

    ...
    @gensym closure T
    func[:body] = quote
        let $closure() = $(func[:body]), $T = $(Base).promote_op($closure)
            if !$(Base).isconcretetype($T)
                ...
            end

            return $closure()::$T
        end
    end

    return combinedef(func)
end

```

It seems like `promote_op` gets the same results if you put all the function body in a closure. And, as before, the check looks to compile away if everything is type-stable.

---

<div class="post-metadata">

**Author:** ![thofma](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/thofma/32/1691_2.png) [@thofma](https://discourse.julialang.org/u/thofma)\
**Post date:** [May 25, 2024, 6:05pm UTC](https://discourse.julialang.org/t/improving-speed-of-runtime-dispatch-detector/114697/23 "2024-05-25T18:05:50Z")

</div>

I have not tried it, but maybe you can use `@stable module A` to rewire `A.include(x)` to make it default to `A.include(f, x)`, where `f` is a map injecting `@stable` into function definitions?

---

<div class="post-metadata">

**Author:** ![MilesCranmer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/milescranmer/32/21070_2.png) [@MilesCranmer](https://discourse.julialang.org/u/MilesCranmer)\
**Post date:** [May 25, 2024, 6:09pm UTC](https://discourse.julialang.org/t/improving-speed-of-runtime-dispatch-detector/114697/24 "2024-05-25T18:09:13Z")

</div>

Clever! I’ll try that.

---

<div class="post-metadata">

**Author:** ![Elrod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/elrod/32/22461_2.png) [@Elrod](https://discourse.julialang.org/u/Elrod)\
**Post date:** [May 25, 2024, 8:02pm UTC](https://discourse.julialang.org/t/improving-speed-of-runtime-dispatch-detector/114697/25 "2024-05-25T20:02:52Z")

</div>

> [@MilesCranmer](#):
>
> Is there any way to force Julia to specialize no matter what?

You can define something like

```julia
struct StableType{T} end
_typeof(x) = typeof(x)
_typeof(::StableType{T}) where {T} = Type{T}

f(a, t::Type{T}) where {T} = sum(a; init=zero(T))
x = rand(4); ST = StableType{Float64}();
Base.promote_op(f, map(_typeof, (x, ST))...)

```

I get

```julia
julia> Base.promote_op(f, map(_typeof, (x, ST))...)
Float64

```

Of course, also define the standard unwrap that returns just `T` for passing to `f`.

Probably

```julia
makestable(x) = x
makestable(::Type{T}) where {T} = StableType{T}()
unwrapstable(x) = x
unwrapstable(::StableType{T}) where {T} = T

```

then call `stable_wrap(f, map(makestable, args...)...)` and have then have `stable_wrap` itself call `f(map(unwrapsbale, args...)...)`, along with the `Base.promote_op(f, map(_typeof, args)...)`.

---

<div class="post-metadata">

**Author:** ![MilesCranmer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/milescranmer/32/21070_2.png) [@MilesCranmer](https://discourse.julialang.org/u/MilesCranmer)\
**Post date:** [May 25, 2024, 8:17pm UTC](https://discourse.julialang.org/t/improving-speed-of-runtime-dispatch-detector/114697/26 "2024-05-25T20:17:17Z")

</div>

Is there any reason to avoid the closure approach [above](https://discourse.julialang.org/t/improving-speed-of-runtime-dispatch-detector/114697/22)? It seems to work just as well, hugely simplifies the codebase, and also means one can safely use `@stable` on functions without variable names:

```julia
@stable f(x, ::Type{T}) where {T} = ...

```

whereas with a separate wrapper function, this would not be permissible.

---

<div class="post-metadata">

**Author:** ![MilesCranmer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/milescranmer/32/21070_2.png) [@MilesCranmer](https://discourse.julialang.org/u/MilesCranmer)\
**Post date:** [May 25, 2024, 8:21pm UTC](https://discourse.julialang.org/t/improving-speed-of-runtime-dispatch-detector/114697/27 "2024-05-25T20:21:55Z")

</div>

Btw, I got the module-level approach working: [Add support for module-level `@stable` by MilesCranmer · Pull Request #4 · MilesCranmer/DispatchDoctor.jl · GitHub](https://github.com/MilesCranmer/DispatchDoctor.jl/pull/4)

Makes it possible to use on entire codebases:

```julia
@stable module A
    using DispatchDoctor: @unstable

    @unstable f1() = rand(Bool) ? 0 : 1.0
    f2(x) = x
    f3(; a=1) = a > 0 ? a : 0.0

    include("other_code.jl")
end

```

- `@unstable` turns it off (in case you have a function you _need_ to be unstable)
- With @thofma’s idea (thanks!), `include(s)` is automatically mapped to `include(_stable_all_fnc, s)` which propagates the `@stable` through the included code

---

<div class="post-metadata">

**Author:** ![danielwe](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/danielwe/32/35657_2.png) [@danielwe](https://discourse.julialang.org/u/danielwe)\
**Post date:** [May 25, 2024, 9:04pm UTC](https://discourse.julialang.org/t/improving-speed-of-runtime-dispatch-detector/114697/28 "2024-05-25T21:04:19Z")

</div>

Wow, this looks incredibly useful! Some thoughts:

- Currently, neither function-level nor module-level `@stable` apply recursively to functions defined within the scope of other functions. In other words, `g` does not receive the `@stable` treatment in either of the following examples. Is this what you prefer?

```julia
@stable function f(xs)
    g(x) = -2x
    return map(g, xs)
end

```

```julia
@stable module A
    function f(xs)
        g(x) = -2x
        return map(g, xs)
    end
end

```

- Perhaps it would be useful to allow using `@stable` on a begin-end block as well:

```julia
@stable begin
    # stable defs
end

```

I don’t know a lot of metaprogramming, but it looks like this could be achieved simply by replacing `_stable_fnc` with `_stable_all_fnc` in the non-module branch here: [DispatchDoctor.jl/src/DispatchDoctor.jl at main · MilesCranmer/DispatchDoctor.jl · GitHub](https://github.com/MilesCranmer/DispatchDoctor.jl/blob/main/src/DispatchDoctor.jl#L39)

---

<div class="post-metadata">

**Author:** ![thofma](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/thofma/32/1691_2.png) [@thofma](https://discourse.julialang.org/u/thofma)\
**Post date:** [May 25, 2024, 9:10pm UTC](https://discourse.julialang.org/t/improving-speed-of-runtime-dispatch-detector/114697/29 "2024-05-25T21:10:30Z")

</div>

Does it work with nested include?

---

<div class="post-metadata">

**Author:** ![MilesCranmer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/milescranmer/32/21070_2.png) [@MilesCranmer](https://discourse.julialang.org/u/MilesCranmer)\
**Post date:** [May 25, 2024, 9:19pm UTC](https://discourse.julialang.org/t/improving-speed-of-runtime-dispatch-detector/114697/30 "2024-05-25T21:19:38Z")

</div>

> [@danielwe](#):
>
> Currently, neither function-level nor module-level `@stable` apply recursively to functions defined within the scope of other functions. In other words, `g` does not receive the `@stable` treatment in either of the following examples. Is this what you prefer?

Yeah this was on purpose. There’s even a unittest against such behavior here: [DispatchDoctor.jl/test/runtests.jl at 74af05c9f98160c6cdeb7fff246a574604047cf5 · MilesCranmer/DispatchDoctor.jl · GitHub](https://github.com/MilesCranmer/DispatchDoctor.jl/blob/74af05c9f98160c6cdeb7fff246a574604047cf5/test/runtests.jl#L168-L180). Only top-level functions are wrapped, no closures.

> [@danielwe](#):
>
> Perhaps it would be useful to allow using `@stable` on a begin-end block as well:

Since `@stable` only applies to _functions_, I think this may be the wrong pattern to use, as it might give the impression that it detects instability in arbitrary code, which it does not. To me `module` makes more sense because that’s where functions live.

(You should be able to use `@stable` explicitly on a closure function though)

> [@thofma](#):
>
> Does it work with nested include?

It should, since all it does is put

```julia
function include(path::AbstractString)
    return include(_stable_all_fnc, path)
end

```

at the top of the module definition. But note it doesn’t recursively apply to submodules.

---

<div class="post-metadata">

**Author:** ![danielwe](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/danielwe/32/35657_2.png) [@danielwe](https://discourse.julialang.org/u/danielwe)\
**Post date:** [May 25, 2024, 9:42pm UTC](https://discourse.julialang.org/t/improving-speed-of-runtime-dispatch-detector/114697/31 "2024-05-25T21:42:07Z")

</div>

> [@MilesCranmer](#):
>
> But note it doesn’t recursively apply to submodules.

There seem to be a couple of caveats regarding submodules. Firstly, every module and submodule within `@stable` requires its own `using DispatchDoctor`, otherwise you’ll get an error:

```julia-repl
julia> @eval @stable module A
           using DispatchDoctor
           foo(x) = x
           module B
               bar(x) = -x
           end
       end
ERROR: syntax: module expression third argument must be a block
[...]

julia> @eval @stable module A
           using DispatchDoctor
           foo(x) = x
           module B
               using DispatchDoctor
               bar(x) = -x
           end
       end
WARNING: replacing module A.
Main.A

```

Secondly, `@stable` will apply to functions defined in the same file as the `module` statement for the submodule (it just won’t propagate through `include` within the submodule). In the code above, `A.B.bar` gets the treatment:

```julia-repl
julia> @code_lowered A.B.bar(1.0)
CodeInfo(
1 ─ %1 = Main.A.B.:(var"##bar_closure#235#1")
│ %2 = Core.typeof(x)
│ %3 = Core.apply_type(%1, %2)
│ bar_closure#235 = %new(%3, x)
│ %5 = Base.getproperty(Base, :promote_op)
│ %6 = (%5)(bar_closure#235)
│ bar_return_type#236 = %6
│ %8 = Base.getproperty(Base, :isconcretetype)
│ %9 = (%8)(bar_return_type#236)
│ %10 = !%9
└── goto #3 if not %10
2 ─ %12 = Main.A.B.bar
│ %13 = Core.tuple(x)
│ %14 = Core.NamedTuple()
│ %15 = (TypeInstabilityError)(%12, %13, %14, bar_return_type#236)
└── Main.A.B.throw(%15)
3 ┄ %17 = (bar_closure#235)()
│ %18 = Core.typeassert(%17, bar_return_type#236)
└── return %18
)

```

---

<div class="post-metadata">

**Author:** ![danielwe](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/danielwe/32/35657_2.png) [@danielwe](https://discourse.julialang.org/u/danielwe)\
**Post date:** [May 25, 2024, 9:57pm UTC](https://discourse.julialang.org/t/improving-speed-of-runtime-dispatch-detector/114697/32 "2024-05-25T21:57:47Z")

</div>

> [@MilesCranmer](#):
>
> it might give the impression that it detects instability in arbitrary code, which it does not. To me `module` makes more sense because that’s where functions live.

Fair enough! I think applying to begin-end could be practically useful in many cases, say if you want only a subset of functions within a module to be `@stable`, perhaps every method of a function or every constructor for a type. But I get where you’re coming from. However, `@unstable` can be applied to a begin-end block as-is, so there’s a slight asymmetry there.

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [May 25, 2024, 10:06pm UTC](https://discourse.julialang.org/t/improving-speed-of-runtime-dispatch-detector/114697/33 "2024-05-25T22:06:15Z")

</div>

> [@danielwe](#):
>
> Firstly, every module and submodule within `@stable` requires its own `using DispatchDoctor`

The macro could add `using DispatchDoctor: @stable`.

---

<div class="post-metadata">

**Author:** ![danielwe](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/danielwe/32/35657_2.png) [@danielwe](https://discourse.julialang.org/u/danielwe)\
**Post date:** [May 25, 2024, 10:09pm UTC](https://discourse.julialang.org/t/improving-speed-of-runtime-dispatch-detector/114697/34 "2024-05-25T22:09:06Z")

</div>

Actually, it looks like `import DispatchDoctor` is sufficient. No names needed.

---

<div class="post-metadata">

**Author:** ![MilesCranmer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/milescranmer/32/21070_2.png) [@MilesCranmer](https://discourse.julialang.org/u/MilesCranmer)\
**Post date:** [May 25, 2024, 10:40pm UTC](https://discourse.julialang.org/t/improving-speed-of-runtime-dispatch-detector/114697/35 "2024-05-25T22:40:16Z")

</div>

> [@danielwe](#):
>
> Secondly, `@stable` will apply to functions defined in the same file as the `module` statement for the submodule (it just won’t propagate through `include` within the submodule).

Oh yeah I spoke incorrectly. There’s actually no logic to skip submodules, so I guess it will work. It’s only that `include` within submodules won’t apply the transformations. The only time the recursion stops is for `@stable` (to avoid repeats) or `@unstable`. But other than that it will just apply to all function definitions it can find.

Should we add functionality to recursively overload `include` maybe? It should be pretty simple – can just add that overload on any module inside the wrapped expression.

---

<div class="post-metadata">

**Author:** ![MilesCranmer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/milescranmer/32/21070_2.png) [@MilesCranmer](https://discourse.julialang.org/u/MilesCranmer)\
**Post date:** [May 25, 2024, 10:44pm UTC](https://discourse.julialang.org/t/improving-speed-of-runtime-dispatch-detector/114697/36 "2024-05-25T22:44:59Z")

</div>

> [@danielwe](#):
>
> Firstly, every module and submodule within `@stable` requires its own `using DispatchDoctor`, otherwise you’ll get an error:

I’m confused why that error shows up for you, but not in any unit tests… See [DispatchDoctor.jl/test/runtests.jl at 74af05c9f98160c6cdeb7fff246a574604047cf5 · MilesCranmer/DispatchDoctor.jl · GitHub](https://github.com/MilesCranmer/DispatchDoctor.jl/blob/74af05c9f98160c6cdeb7fff246a574604047cf5/test/runtests.jl#L142)

Are you using the latest commit? Could have been a bug from an earlier revision.

---

<div class="post-metadata">

**Author:** ![danielwe](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/danielwe/32/35657_2.png) [@danielwe](https://discourse.julialang.org/u/danielwe)\
**Post date:** [May 25, 2024, 10:50pm UTC](https://discourse.julialang.org/t/improving-speed-of-runtime-dispatch-detector/114697/37 "2024-05-25T22:50:45Z")

</div>

> [@MilesCranmer](#):
>
> Are you using the latest commit?

Yes. In a fresh Julia session:

```julia-repl
(@v1.10) pkg> activate --temp
  Activating new project at `/tmp/jl_AK8zpN`

(jl_AK8zpN) pkg> add https://github.com/MilesCranmer/DispatchDoctor.jl
    Updating git-repo `https://github.com/MilesCranmer/DispatchDoctor.jl`
   Resolving package versions...
    Updating `/tmp/jl_AK8zpN/Project.toml`
  [8d63f2c5] + DispatchDoctor v0.1.0 `https://github.com/MilesCranmer/DispatchDoctor.jl#main`
    Updating `/tmp/jl_AK8zpN/Manifest.toml`
  [8d63f2c5] + DispatchDoctor v0.1.0 `https://github.com/MilesCranmer/DispatchDoctor.jl#main`
  [1914dd2f] + MacroTools v0.5.13
  [1c621080] + TestItems v0.1.1
  [2a0f44e3] + Base64
  [d6f4376e] + Markdown
  [9a3f8284] + Random
  [ea8e919c] + SHA v0.7.0

julia> using DispatchDoctor

julia> @eval @stable module A
           import DispatchDoctor
           foo(x) = x
           module B
               bar(x) = -x
           end
       end
ERROR: syntax: module expression third argument must be a block
Stacktrace:
 [1] eval(m::Module, e::Any)
   @ Core ./boot.jl:385
 [2] top-level scope
   @ REPL[4]:1

```

Same if I put the code, sans `@eval`, in a file and use `include`.

* * *

```julia-repl
julia> versioninfo()
Julia Version 1.10.3
Commit 0b4590a5507 (2024-04-30 10:59 UTC)
Build Info:
  Official https://julialang.org/ release
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 8 × Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-15.0.7 (ORCJIT, skylake)
Threads: 1 default, 0 interactive, 1 GC (on 8 virtual cores)

```

---

<div class="post-metadata">

**Author:** ![MilesCranmer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/milescranmer/32/21070_2.png) [@MilesCranmer](https://discourse.julialang.org/u/MilesCranmer)\
**Post date:** [May 25, 2024, 10:53pm UTC](https://discourse.julialang.org/t/improving-speed-of-runtime-dispatch-detector/114697/38 "2024-05-25T22:53:35Z")

</div>

Oh, you just mean submodules? But the top-level module doesn’t need that, right?

---

<div class="post-metadata">

**Author:** ![danielwe](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/danielwe/32/35657_2.png) [@danielwe](https://discourse.julialang.org/u/danielwe)\
**Post date:** [May 25, 2024, 10:55pm UTC](https://discourse.julialang.org/t/improving-speed-of-runtime-dispatch-detector/114697/39 "2024-05-25T22:55:27Z")

</div>

I’m afraid I need it at top level too:

```julia-repl
julia> @eval @stable module C
           foo(x) = x
       end
ERROR: LoadError: AssertionError: module_body.head == :block
Stacktrace:
 [1] _stable_module(ex::Expr)
   @ DispatchDoctor ~/.julia/packages/DispatchDoctor/J9g5M/src/DispatchDoctor.jl:47
 [2] _stable(ex::Expr)
   @ DispatchDoctor ~/.julia/packages/DispatchDoctor/J9g5M/src/DispatchDoctor.jl:37
 [3] var"@stable"( __source__ ::LineNumberNode, __module__ ::Module, fex::Any)
   @ DispatchDoctor ~/.julia/packages/DispatchDoctor/J9g5M/src/DispatchDoctor.jl:136
 [4] eval(m::Module, e::Any)
   @ Core ./boot.jl:385
 [5] top-level scope
   @ REPL[9]:1
in expression starting at REPL[9]:1

julia> @eval @stable module C
           import DispatchDoctor
           foo(x) = x
       end
Main.C

```

---

<div class="post-metadata">

**Author:** ![MilesCranmer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/milescranmer/32/21070_2.png) [@MilesCranmer](https://discourse.julialang.org/u/MilesCranmer)\
**Post date:** [May 25, 2024, 10:57pm UTC](https://discourse.julialang.org/t/improving-speed-of-runtime-dispatch-detector/114697/40 "2024-05-25T22:57:04Z")

</div>

Oh I think it’s not the import, but rather the fact that your module is a single line of code — the macro expects it to be a block. If you have any import there it should work. So it’s a bug but unrelated to the import (I think)

[Previous page](https://discourse.julialang.org/t/improving-speed-of-runtime-dispatch-detector/114697.md?page=1)

[Next page](https://discourse.julialang.org/t/improving-speed-of-runtime-dispatch-detector/114697.md?page=3)