Proposal for SharedFunctions.jl package for optional dependency management

Internals & Design
66

There are three several topics here as I see it and I’ll do my best to be brief and concise, with as little errors as possible

  1. The discussion started on having the need to define some abstract base module with names, so different modules can coordinate upon(extend common interface)

  2. It evolved to:

  1. @tkf made the observation that we can extend a function in a module before it is loaded, in a fashion that is roughly equivalent to the following:
    let
    Function{UID,NAME} where UID where NAME
    be the abstract base type of all functions in julia, where UID stands for the module or package UUID
    Then any module can extend a function in another module without having to specifically load it.

Here is an example of extending function plot in module Plots using module name for UID

(plot::Function{:Plots,:plot}(x::MyVecType) = plot(toBaseVec(x))

  1. Jeff suggested that there will be a way to tell the system to merge functions in different modules(hope I understood correctly)

  2. @chakravala wants to do the same but make the merge visible only locally as not to affect the integrity of other modules

  3. Context Dispatch is a generalization of the way multiple-dispatch is handled today as it enables you to do the same things, using the same code, however, an implicit merging of exported function names as in 5 feels very natural in this way and can often replace explicit coordination as in 3

so for your toy example, calling function B.f from the context of module C uses modules Base, A and B for its method table so there is no problem.

Now let’s say module D who uses C, and calls C.g() which in turn calls B.f with the constructed types.
Since it can be proved that the context of C will suffice, the context can be narrowed.

And as long as you don’t change the state of module C or its dependent modules you can re-use the
binary resulting from compiling the code for call C.g() from the context of C

Another aspect is that the automatic merging of exported names suddenly looks very appealing.

Your example with Context Dispatch and automatic merging of exported names would look like:

module Base
    sort!(x,y) = isless(x,y)
    isless(x,y) = ErrorException("unimplemented")
    export isless,sort!
end

module A
    struct T end
    isless(x::T,y::T) = true
    export isless
end

module B
    using Base
    f(x,y) = sort!(x,y)
end

module C
    using Base, A, B
    
    g() = B.f(A.T(),A.T())
end

In the context of module C, the functions Base.isless and A.isless got merged.

I don’t think so, like many others here, I think, I learned about compilers and LLVM and multiple-dispatch through the feat of engineering we call “julia”.

It came up initially to solve what @chakravala was writing about, it was a little bit obscure in the beginning but it starts to feel more and more like the right way to go.

O.k it took me more than 3 hours to write this, and I am losing focus, what I meant by the saying
“I don’t own the idea” is that it feels like a larger subject than just a small eureka moment.

11 Likes