# Discussion: Context Dispatch - yes? no? questions? answers?

**URL:** <https://discourse.julialang.org/t/discussion-context-dispatch-yes-no-questions-answers/24194>\
**Category:** Internals & Design\
**Created:** [May 14, 2019, 6:04am UTC](https://discourse.julialang.org/t/discussion-context-dispatch-yes-no-questions-answers/24194 "2019-05-14T06:04:01Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![TsurHerman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tsurherman/32/1234_2.png) [@TsurHerman](https://discourse.julialang.org/u/TsurHerman)\
**Post date:** [May 14, 2019, 6:04am UTC](https://discourse.julialang.org/t/discussion-context-dispatch-yes-no-questions-answers/24194/1 "2019-05-14T06:04:01Z")

</div>

I wonder what the devs and community think of the direction of context dispatch as it has been recently presented in other various posts.

---

<div class="post-metadata">

**Author:** ![rfourquet](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rfourquet/32/3610_2.png) [@rfourquet](https://discourse.julialang.org/u/rfourquet)\
**Post date:** [May 14, 2019, 6:45am UTC](https://discourse.julialang.org/t/discussion-context-dispatch-yes-no-questions-answers/24194/2 "2019-05-14T06:45:20Z")

</div>

What do you mean by “context dispatch” ?

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [May 14, 2019, 7:09am UTC](https://discourse.julialang.org/t/discussion-context-dispatch-yes-no-questions-answers/24194/3 "2019-05-14T07:09:24Z")

</div>

It would be great if you could at least link the relevant posts, or ideally write up a coherent, detailed proposal. Reviewing something scattered over “various posts” is not ideal.

---

<div class="post-metadata">

**Author:** ![felipenoris](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/felipenoris/32/553_2.png) [@felipenoris](https://discourse.julialang.org/u/felipenoris)\
**Post date:** [May 14, 2019, 12:08pm UTC](https://discourse.julialang.org/t/discussion-context-dispatch-yes-no-questions-answers/24194/4 "2019-05-14T12:08:34Z")

</div>

If you meant ‘contextual dispatch’, this is what allows you to override existing functions in the context of GPU processing. So it has its applications.

```julia
@context GPU
@contextual(GPU) sin(x::Float32) =
 ccall((:__nv_sinf, :libdevice), Cfloat, (Cfloat,) x)

```

---

<div class="post-metadata">

**Author:** ![TsurHerman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tsurherman/32/1234_2.png) [@TsurHerman](https://discourse.julialang.org/u/TsurHerman)\
**Post date:** [May 14, 2019, 1:05pm UTC](https://discourse.julialang.org/t/discussion-context-dispatch-yes-no-questions-answers/24194/5 "2019-05-14T13:05:22Z")

</div>

Hi, sorry, for reference here is the latest discussion we had on this.

> [@Proposal for SharedFunctions.jl package for optional dependency management](https://discourse.julialang.org/t/proposal-for-sharedfunctions-jl-package-for-optional-dependency-management/23526/66):
>
> There are three several topics here as I see it and I’ll do my best to be brief and concise, with as little errors as possible The discussion started on having the need to define some abstract base module with names, so different modules can coordinate upon(extend common interface) It evolved to: @tkf made the observation that we can extend a function in a module before it is loaded, in a fashion that is roughly equivalent to the following: let Function{UID,NAME} where UID where N…

---

<div class="post-metadata">

**Author:** ![thautwarm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/thautwarm/32/37760_2.png) [@thautwarm](https://discourse.julialang.org/u/thautwarm)\
**Post date:** [May 14, 2019, 6:58pm UTC](https://discourse.julialang.org/t/discussion-context-dispatch-yes-no-questions-answers/24194/6 "2019-05-14T18:58:59Z")

</div>

The answer should be yes, or in some case you cannot extension things at all.  
If you re-extension some already existing methods, you do type piracy. However, something to achieve the functionalities(of new extension for some argument type) might be a hard demand, which comes up with a requirement about scoped method dispatch aka an instance of the contextual dispatch.

---

<div class="post-metadata">

**Author:** ![jpsamaroo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jpsamaroo/32/46804_2.png) [@jpsamaroo](https://discourse.julialang.org/u/jpsamaroo)\
**Post date:** [May 14, 2019, 8:17pm UTC](https://discourse.julialang.org/t/discussion-context-dispatch-yes-no-questions-answers/24194/7 "2019-05-14T20:17:50Z")

</div>

If you’re referring to the ability to “safely” add dispatches that would normally be considered type piracy, then the widely-understood “contextual dispatch” (aka Cassette or IRTools) is exactly what you want. The real issue with regular type piracy is that it can cause unexpected behavior by just loading a package; if such a change in behavior instead required a call to `Cassette.overdub` to occur, then it wouldn’t be type piracy at all (at least not in the detrimental sense).

In other words, we already have this and it works quite well!

---

<div class="post-metadata">

**Author:** ![jpsamaroo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jpsamaroo/32/46804_2.png) [@jpsamaroo](https://discourse.julialang.org/u/jpsamaroo)\
**Post date:** [May 14, 2019, 8:22pm UTC](https://discourse.julialang.org/t/discussion-context-dispatch-yes-no-questions-answers/24194/8 "2019-05-14T20:22:29Z")

</div>

Would you mind briefly reiterating what your form of “contextual dispatch” implies? The sharing of this term with what Cassette-like packages implement makes things rather confusing (maybe you can come up with an alternative term for it?). Some things I’d like to have explicitly stated include:

- How it differs from the equivalent term used to describe how Cassette et. al work, i.e. “overdubbing” or “user-defined passes”.
- What problems it’s intended to solve that aren’t already solved well via a different approach (such as with Cassette).
- What will be required to implement it, and where it will need to be implemented (can it be done in a package, or do the changes need to be to Julia’s core?).
- Has anything changed w.r.t your “contextual dispatch” in the months since your original post? EDIT: Here’s the post I’m referring to: [MultiFunctions: Context dispatch and binary cache](https://discourse.julialang.org/t/multifunctions-context-dispatch-and-binary-cache/11565)

---

<div class="post-metadata">

**Author:** ![TsurHerman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tsurherman/32/1234_2.png) [@TsurHerman](https://discourse.julialang.org/u/TsurHerman)\
**Post date:** [May 15, 2019, 4:36am UTC](https://discourse.julialang.org/t/discussion-context-dispatch-yes-no-questions-answers/24194/9 "2019-05-15T04:36:55Z")

</div>

It is very similar to ”Context Dispatch” of cassette but is aimed to be implemented on the language level.  
It would help solve a few current pain points of the language.

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [May 16, 2019, 4:25am UTC](https://discourse.julialang.org/t/discussion-context-dispatch-yes-no-questions-answers/24194/10 "2019-05-16T04:25:18Z")

</div>

> [@Proposal for SharedFunctions.jl package for optional dependency management](https://discourse.julialang.org/t/proposal-for-sharedfunctions-jl-package-for-optional-dependency-management/23526/66):
>
> **The building blocks:**
> 
> - A Context is a set of modules.
> - The Context of module M is the set of all modules that module M depends upon.
> - Extension: The method table for a Context consists only from methods belonging to modules in the Context
> - Propagation: the context is propagated recursively down the AST, and is narrowed if proven that it will resolve the call just as specific as the wider context
> 
> […]
> 
> so for your toy example, calling function B.f from the context of module C uses modules Base, A and B for its method table so there is no problem.

This seems like it requires a new version of `sort!` to be compiled not only for each type signature (roughly what we currently do), but also for each type signature and for each calling context since the method tables of all functions that are called are different in different contexts. This seems like it would massively increase the amount of compilation required, not decrease it.

> [@Proposal for SharedFunctions.jl package for optional dependency management](https://discourse.julialang.org/t/proposal-for-sharedfunctions-jl-package-for-optional-dependency-management/23526/66):
>
> Your example with Context Dispatch and automatic merging of exported names would look like:
> 
> ```julia
> module Base
> sort!(x,y) = isless(x,y)
> isless(x,y) = ErrorException("unimplemented")
> export isless,sort!
> end
> 
> module A
> struct T end
> isless(x::T,y::T) = true
> export isless
> end
> 
> module B
> using Base
> f(x,y) = sort!(x,y)
> end
> 
> module C
> using Base, A, B
>     
> g() = B.f(A.T(),A.T())
> end
> 
> ```
> 
> In the context of module C, the functions Base.isless and A.isless got merged.

What if module `A` instead defined

```julia
isless(x, y) = "surprise!"

```

This seems like it would cause `sort!` to fail with a method error even though the two `isless` functions are not the same generic function and Base is not trying to call the one that `A` defines or exports.

---

<div class="post-metadata">

**Author:** ![TsurHerman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tsurherman/32/1234_2.png) [@TsurHerman](https://discourse.julialang.org/u/TsurHerman)\
**Post date:** [May 16, 2019, 9:08am UTC](https://discourse.julialang.org/t/discussion-context-dispatch-yes-no-questions-answers/24194/11 "2019-05-16T09:08:00Z")

</div>

> [@StefanKarpinski](#):
>
> This seems like it requires a new version of `sort!` to be compiled not only for each type signature (roughly what we currently do), but also for each type signature and for each calling context since the method tables of all functions that are called are different in different contexts

That is generally correct unless some heuristic is inserted for narrowing the contexts, however, even in the general un-optimized case, all of the compilation output is cachable and reusable.

Whereas currently in every Julia session `sort!` is being compiled again and again.  
So overall Context Dispatch even in its space wasting general form, would still decrease the amount spent on compiling drastically.

> [@StefanKarpinski](#):
>
> What if module `A` instead defined
> 
> ```julia
> isless(x, y) = "surprise!"
> 
> ```

Ah, very interesting, that is not directly relevant to Context Dispatch but to the complementary idea -Automatic Merging of Exported Functions.  
And in that case you should get a Method Error: ambiguous just like you would get now.

for example in the following code:

```julia
struct A
    a::Int
end

Base.isless(x,y::Main.A) = Base.isless(x.a,y.a)
Base.isless(x::Main.A,y) = !Base.isless(x.a,y.a)

vecA = [A(1) for i=1:100]
sorted = sort(vecA) #MethodError: isless(::A, ::A) is ambiguous.

```

---

<div class="post-metadata">

**Author:** ![chakravala](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chakravala/32/6832_2.png) [@chakravala](https://discourse.julialang.org/u/chakravala)\
**Post date:** [May 17, 2019, 4:20am UTC](https://discourse.julialang.org/t/discussion-context-dispatch-yes-no-questions-answers/24194/12 "2019-05-17T04:20:33Z")

</div>

> [@TsurHerman](#):
>
> > [@StefanKarpinski](#):
> >
> > What if module `A` instead defined
> > 
> > ```julia
> > isless(x, y) = "surprise!"
> > 
> > ```
> > 
> > This seems like it would cause `sort!` to fail with a method error even though the two `isless` functions are not the same generic function and Base is not trying to call the one that `A` defines or exports.
> 
> Ah, very interesting, that is not directly relevant to Context Dispatch but to the complementary idea -Automatic Merging of Exported Functions.  
> And in that case you should get a Method Error: ambiguous just like you would get now.

This is an issue that was already discussed before, in terms of the `SharedFunctions` discussion

> [@Proposal for SharedFunctions.jl package for optional dependency management](https://discourse.julialang.org/t/proposal-for-sharedfunctions-jl-package-for-optional-dependency-management/23526/28):
>
> A different generic function with the same name is a function with the same name as another function… For example:
> 
> ```julia
> module A
> f(x) = x
> end
> 
> module B
> f(x, y) = x + y
> end
> 
> ```
> 
> Here `A.f` and `B.f` are different (generic) functions.
> 
> The reason why we cannot just merge the two `f` is because of how you write generic code.

> [@Proposal for SharedFunctions.jl package for optional dependency management](https://discourse.julialang.org/t/proposal-for-sharedfunctions-jl-package-for-optional-dependency-management/23526/31):
>
> How about this rule to solve this: Only import a name from a SharedFunctions package if you don’t intend to write generic method with only the Any type dispatch; otherwise have the generic definitions in SharedFunctions Shared function names with completely generic method definitions should either have the generic definitions in the SharedFunctions package not be imported from SharedFunctions This way, if there is a need for generic methods, the fully generic method is either shared by al…

> [@Proposal for SharedFunctions.jl package for optional dependency management](https://discourse.julialang.org/t/proposal-for-sharedfunctions-jl-package-for-optional-dependency-management/23526/34):
>
> In theory we could almost make it automatic, right? That is, have a github bot that generates a PR that extracts all exported function names, abstract types and abstract docstrings into an abstract header package, and rewires the old package to require and import and extend and reexport functions from the abstract header package. In theory, the creation of lightweight AbstractFoo / HeaderFoo packages from some Foo package should not requite a lot of human thought or intervention (corner case: …