# Julep: Taking multiple dispatch,export,import,binary compilation seriously

**URL:** <https://discourse.julialang.org/t/julep-taking-multiple-dispatch-export-import-binary-compilation-seriously/10882>\
**Category:** Internals & Design\
**Tags:** namespaces\
**Created:** [May 14, 2018, 9:58am UTC](https://discourse.julialang.org/t/julep-taking-multiple-dispatch-export-import-binary-compilation-seriously/10882 "2018-05-14T09:58:33Z")\
**Posts on this page:** 20\
**Page:** 4

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [May 16, 2018, 3:27pm UTC](https://discourse.julialang.org/t/julep-taking-multiple-dispatch-export-import-binary-compilation-seriously/10882/61 "2018-05-16T15:27:14Z")

</div>

> [@TsurHerman](#):
>
> I trust your knowledge and experience , but I also trust mine … so in the words of big lebovsky “thats like your’e opinion man”

So basically you treat all non-techincal admiration as proof that what you are doing is in the right direction and you treat all detailed technical explaination as “just your opinion”. I see the difference between “criticise freely” and actually accepting critisism now.

Now back to the more techinical part.

> [@TsurHerman](#):
>
> yes once, but only once … unless something change in the module it is defined in, just like any .so .dylib .dll out there.

Shared libraries are useful because they are compiled **AHEAD OF TIME**. How many times something is compiled does **NOT** matter. For what it worth, a specialization of a method in a certain world range will already only be compiled once (ok if it’s inlined that’s slightly different but the whole point of inlining is to compile multiple times). What you propose will won’t help even compiling things once since

1. There isn’t any problem of compiling multiple times to begin with so nothing can help.
2. Everytime you imported something the whole lookup will need to be redone, exactly the same as what it is now.

> [@TsurHerman](#):
>
> I know this may seem added verbosity , but it feels more predictable to the end user.

Absolutely not. What you are saying is that there should be no name isolation, no implementation detail. All the implementation detail has to be exported and defeating what you are trying to do anyway.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [May 16, 2018, 4:03pm UTC](https://discourse.julialang.org/t/julep-taking-multiple-dispatch-export-import-binary-compilation-seriously/10882/62 "2018-05-16T16:03:33Z")

</div>

> [@TsurHerman](#):
>
> it is then encounters Base.start which is an exported function therefore according the rules I layed out(which may be that better ones exist…) it is resolved to function in its scope (Base) and in the calling scope which is B (remember we called B.sum which has Base.sum as one of its methods)

Except that you don’t want `sum` to call `Base.start` or `B.start`, you want to it to call `A.start` (because you are summing an `A.T()` object, even though you are not calling _from_ module `A`.)

(If your answer is that the `A.start` method was merged with `B.start` and `Base.start` when B imported A, then you’ve arrived essentially back at the current semantics: if you import module `A` _anywhere_ in your program, then `A.start` needs to be visible _everywhere_ in your program, because an `x = A.T()` object could be passed to _any scope_ in your program, and that scope could call e.g. `Base.sum(x)`.)

---

<div class="post-metadata">

**Author:** ![ScottPJones](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/scottpjones/32/146_2.png) [@ScottPJones](https://discourse.julialang.org/u/ScottPJones)\
**Post date:** [May 16, 2018, 4:06pm UTC](https://discourse.julialang.org/t/julep-taking-multiple-dispatch-export-import-binary-compilation-seriously/10882/63 "2018-05-16T16:06:33Z")

</div>

> [@ChrisRackauckas](#):
>
> Or Cassette’s context-specific dispatching could likely make it easy to come up with implementations of these kinds of things.

@jrevels talk last night on [Cassette.jl](https://github.com/jrevels/Cassette.jl) was great (it seems so magical!) and Jeff had some interesting ideas on how to use it for debugging (which was great news, since [Traceur.jl](https://github.com/MikeInnes/Traceur.jl) is broken on master (not it’s fault, [ASTInterpreter2.jl](https://github.com/Keno/ASTInterpreter2.jl) doesn’t load on master).  
I’m going to investigate Cassette later today)

---

<div class="post-metadata">

**Author:** ![jeff.bezanson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeff.bezanson/32/48_2.png) [@jeff.bezanson](https://discourse.julialang.org/u/jeff.bezanson)\
**Post date:** [May 16, 2018, 4:19pm UTC](https://discourse.julialang.org/t/julep-taking-multiple-dispatch-export-import-binary-compilation-seriously/10882/64 "2018-05-16T16:19:15Z")

</div>

> [@TsurHerman](#):
>
> Resolving dispatch can be defined recursively upwards in the AST (toward the caller) as follows:  
> if `func` is not an exported function it is resolved in its Module (according to B).  
> If it is an exported function it is resolved in its module ,and in its calling function scope

If I understand correctly, you’re proposing looking at the call stack and using the module of the calling code. If so, mentioning the “AST” here is incorrect, as the call stack is a dynamic property, not a static/syntactic property. You cannot determine the caller of a function by looking at ASTs.

This is closely related to the notion of “dynamic scope” (as opposed to lexical scope), where variables are looked up in calling scopes. A general problem with such approaches is that they make it hard to reason about code. If I write

```julia
f() = sum([1,2,3])

```

in my package, I intend for `f()` to return `6`. In your proposal, unless I misunderstand it, I have no idea what `f` does. If it’s called from a package that defines `sum(x) = 42`, or `a+b = 42`, then `f()` will return `42`. Yes, currently it’s possible to overwrite `Base.sum` or `+` of integers, but we discourage that and print a warning if you do it. But in your proposal it would just be the correct behavior to let another package replace the `sum` or `+` you intended to call.

This proposal also exposes implementation details. In the example in your second post, `M` needs to add `using StaticArrays` _because N uses static arrays_. That means if N decides to change its implementation to use StaticArrays, that breaks all of its users. All of its users need to add `using StaticArrays` in order to work again. I believe that is unacceptably non-modular and makes this proposal unworkable.

---

<div class="post-metadata">

**Author:** ![jlperla](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jlperla/32/34332_2.png) [@jlperla](https://discourse.julialang.org/u/jlperla)\
**Post date:** [May 16, 2018, 4:45pm UTC](https://discourse.julialang.org/t/julep-taking-multiple-dispatch-export-import-binary-compilation-seriously/10882/65 "2018-05-16T16:45:20Z")

</div>

> [@stevengj](#):
>
> is a property of nearly all languages with [first-class functions](https://en.wikipedia.org/wiki/First-class_function). (This excludes many traditional static OOP languages like Java and most of C++. C++ and Java

Also python, object-oriented matlab, anything with single-dispatch or operator overloading, etc. Basically almost any other language users will come from. That isn’t to say that the way Julia has done it is wrong, just that unless we can all find a way to tightly communicate the underlying mechanics this will keep coming up. People are not used to the power (and consequences) of first-class functions, even if they have dealt with function overloading and dynamic dispatching.

> [@stevengj](#):
>
> Evaluating the function as an object separate from evaluating the arguments, and then applying the function to the arguments, is a property of nearly all languages with [first-class functions](https://en.wikipedia.org/wiki/First-class_function). (

Yes, this is exactly what needs to be communicated! The way Julia has done namespaces is the natural way to add them to a language with first-class functions… although I don’t know how many languages with first class functions have multiple-dispatch and namespaces. It seems that most languages with first-class functions avoid namespaces, or have similar issues far worse than Julia (e.g. [Anonymous records. A solution to the problems of record-system. · GitHub](https://gist.github.com/nikita-volkov/6977841) )

The way that someone thinking about extending multiple-dispatch as a generalization of single-dispatch is different. Are they reconcilable? No one knows. Is an implementation feasible? Not sure, and it isn’t the time to find out. ADL is the only hope for something that would extend the current semantics and let people better segment namespaces of methods (essentially by have no namespaces for functions). Those questions can be pushed off for now.

> [@stevengj](#):
>
> ake a step back and realize that you are insisting that everyone _else_ is confused because your unfamiliarity with first-class functions made it difficult to communicate with you.

If that is your impression, then it is mistaken. I have tried to say that **everyone** has been confused on this topic, myself likely the most confused of all through most of it (though I think I finally “get it”). I had suggested from your previous summary of the ADL thread (which talked about multi-dispatch) that you stopped reading and not fully digested the conversation, but I apologize if you had.

> [@stevengj](#):
>
> As @jeff.bezanson said [in the other thread](https://discourse.julialang.org/t/function-name-conflict-adl-function-merging/10335/200), even if you have a kind of “ADL object”, you would still need a global namespace of such objects, and furthermore having a method [become applicable because its signature narrowed](https://discourse.julialang.org/t/function-name-conflict-adl-function-merging/10335/204) (and thus entered a new type’s namespace) is totally different from the subtyping model of dispatch. I don’t think you grasped the implications of these remarks, especially the latter. What you are suggesting _is_ a complete upheaval in how the language works, not merely an incremental change (excluding add-on packages that emulate ADL by implementing their own lookup tables).

As @tkoolen said, the later turns out not to be true. If ADL works, it is a super-set of the semantics (where all(?) existing code could work). No changes to the subtyping model of dispatch, which would be a dealbreaker. Again, even if it is a strict superset of the existing semantics, that doesn’t mean it feasible to implement. But that is a different explanation for not having ADL that it being fundamentally incompatible with the language.

> [@stevengj](#):
>
> (which is common to all all other dynamic multiple-dispatch languages AFAIK)

I think it would be very useful to have a list of those multi-dispatch languages that actually implement namespaces. I went on a hunt a few weeks ago and couldn’t find any precedent for Julia’s approach (although I was told CLOS does?). This may not be surprising because Julia is an unprecedented language on many other levels, and most dynamic languages either have skipped namespaces entirely or have single-dispatch (or emulate operator-overloading with single-dispatch namespace lookups). But this sort of thing can wait.

> [@stevengj](#):
>
> It may be interesting to think about as an academic exercise, but I think you’ve confused people understanding _how to better explain_ the difference between Julia’s dispatch and the ADL you are used to with agreeing that there is something _wrong_ with Julia’s approach (which is common to all all other dynamic multiple-dispatch languages AFAIK) that we need to plan to “fix” in Julia 2.0.

Many of us have said, again and again, that we want to push off the any revisiting of Julia 2.0 and ADL. Nobody wants to discuss a deign. The reason this keeps coming up are the responses that core developers give on Julia Discourse, which we think makes the issue worse by telling people that this is a conseqnewce of multi-dispatch rather than first-class functions (he real issue).

To roughly paraphrase the current responses from Julia developers on most of the questions on lookups and namesapces:

> Julia does namespaces this way because it is a language with multi-dispatch, and you are used to single-dispatch. You will learn to deal with what you perceive as downsides, because this is the only way to get all the wonderful features of multi-dispatch.

As discussed in this thread and the ADL thread, this is false and confusing to people from other languages that don’t see what namespaces have to do with multi-dispatch. The real reason is that Julia chose to do namespaces this way is because it is the natural way to add namespaces to a language with first-class functions.

There is also the following corollary, which comes up in various forms more often than I would like

> So learn to live with it, or fork it and get your own users (ed: apologies for the previous explicit)

Luckily this is not from the vast majority of developers, so I will dismiss it.

> [@stevengj](#):
>
> is very far down on everyone’s priority list.

Yes, on everyone’s list. But a coherent way to communicate the current logic of Julia to those coming from other languages is important now, as is a set of macros to make the transition easier. I, personally, would love to see some simple macros as part of the core standard library to make people be more explicit when trying to merge methods.

Can I propose that a few people work on a PR to try to explain this over the next few months, with the intention that it coudl go into the documentation (in the differences from other languages section)? Then anytime these quetsions come up on Discourse, you can point them to people. I also think there are some natural macros, maybe starting from [GitHub - chakravala/ForceImport.jl: Macro that force imports conflicting methods in modules](https://github.com/chakravala/ForceImport.jl) that should be considered as part of the library. Formalizing on this would help everyone to notice the “mild” type piracy inherent in the current design.

---

<div class="post-metadata">

**Author:** ![jeff.bezanson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeff.bezanson/32/48_2.png) [@jeff.bezanson](https://discourse.julialang.org/u/jeff.bezanson)\
**Post date:** [May 16, 2018, 4:54pm UTC](https://discourse.julialang.org/t/julep-taking-multiple-dispatch-export-import-binary-compilation-seriously/10882/66 "2018-05-16T16:54:22Z")

</div>

> [@jlperla](#):
>
> In single-dispatch languages, the basic lookup rule is that whatever the namespace of the type you use as the first argument determines the relevant set of methods for a given function. The natural generalization of this is that with multi-dispatch, is that the namespace **s** of the type **s** you have as the arguments determines the relevant set of methods for a given function.

The key phrase here is “for a given function”. The entire problem is how to identify which function you’re talking about. For example, if a function is just identified by a name (as it is in class-based OO languages) then the rest of your proposal goes through no problem—we could implement it e.g. by having a single namespace of functions.

---

<div class="post-metadata">

**Author:** ![jlperla](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jlperla/32/34332_2.png) [@jlperla](https://discourse.julialang.org/u/jlperla)\
**Post date:** [May 16, 2018, 5:02pm UTC](https://discourse.julialang.org/t/julep-taking-multiple-dispatch-export-import-binary-compilation-seriously/10882/67 "2018-05-16T17:02:28Z")

</div>

> [@jeff.bezanson](#):
>
> The key phrase here is “for a given function”. The entire problem is how to identify which function you’re talking about.

Exactly! People coming from other languages tend to think of types and methods having namespaces and functions being a string of identifiers with no intrinsic namespace. This is not how it works in Julia. If it is possible to have functions in a global namespace and methods in the local namespaces is an implementation question that can wait. I don’t know if ADL can work with Julia, and I don’t want to think too much now about it (as long as we consistently communicate to the community as a whole and training to explain the differences).

---

<div class="post-metadata">

**Author:** ![ExpandingMan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/expandingman/32/866_2.png) [@ExpandingMan](https://discourse.julialang.org/u/ExpandingMan)\
**Post date:** [May 16, 2018, 5:07pm UTC](https://discourse.julialang.org/t/julep-taking-multiple-dispatch-export-import-binary-compilation-seriously/10882/68 "2018-05-16T17:07:33Z")

</div>

> [@jlperla](#):
>
> Can I propose that a few people work on a PR to try to explain this over the next few months, with the intention that it coudl go into the documentation (in the differences from other languages section)?

I, for one, would appreciate a very succinct explanation of what the main issue is and possible resolutions. I’ve been following these discussions off-and-on, and I’ve tried to be open minded as I definitely think there is a real issue here, but to be honest I still can’t help but see it as a very minor one. I come from about 8 years of C++ and FORTRAN (2 years of Julia now) and I have never once gotten really frustrated over any function-namespace-method-extension problem. I think the worst thing that’s ever happened to me is that I declared a function using a name that was already in use, but in every case not only was this easy to resolve but it was obvious how to resolve it, and in many of those cases the name choice was what I’d consider a fairly obvious screw-up on my part anyway. By now I have written a pretty hefty amount of both Julia “package code” and scripts, so it’s getting hard to think that I’m some kind of special case.

It is probably worth mentioning that one place where my view _has_ evolved is that I now understand that there is a spectrum of package types with packages that shouldn’t export anything at one end (e.g. CSV, Feather) and packages that are most useful when they fundamentally alter your namespace on the other (e.g. DifferentialEquations, LightGraphs, JuMP… LinearAlgebra is almost _too_ good an example to count). So, I don’t know, maybe there is some way of formalizing or documenting that concept.

At the risk of sounding too dismissive, I’m starting to think that the rest of this is primarily a documentation issue.

---

<div class="post-metadata">

**Author:** ![Jean\_Michel](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jean_michel/32/8282_2.png) [@Jean\_Michel](https://discourse.julialang.org/u/Jean_Michel)\
**Post date:** [May 16, 2018, 5:15pm UTC](https://discourse.julialang.org/t/julep-taking-multiple-dispatch-export-import-binary-compilation-seriously/10882/69 "2018-05-16T17:15:28Z")

</div>

> [@jlperla](#):
>
> I think it would be very useful to have a list of those multi-dispatch languages that actually implement namespaces

GAP has multiple dispatch and does not really have namespaces. Each function is in the global namespace  
and methods are separated by their signature (by default they are protected from overwriting, you must  
explicitely unprotect a method with its signature to overwrite it). To have a local namespace, a current practice is to put functions as fields in a record (similar to values in a dict with symbol keys: the  
syntax is MyPackage.foo, with meaning similar to MyPackage[:foo]). But then _every_ call must of course be qualified.

---

<div class="post-metadata">

**Author:** ![jlperla](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jlperla/32/34332_2.png) [@jlperla](https://discourse.julialang.org/u/jlperla)\
**Post date:** [May 16, 2018, 5:16pm UTC](https://discourse.julialang.org/t/julep-taking-multiple-dispatch-export-import-binary-compilation-seriously/10882/70 "2018-05-16T17:16:54Z")

</div>

> [@ExpandingMan](#):
>
> At the risk of sounding too dismissive, I’m starting to think that the rest of this is primarily a documentation issue.

I think that is a reasonable perspective. I would say that there are separate things:

1. Consistent communication and documentation when all these things come up, taking into account the perspective of most users (who do not know about first-class functions)
2. Macros which may make the current workarounds of merging operators into `Base`, etc. more explicit and easier to teach
3. A discussion in a year or two on whether a modified version of the lookup rules could significantly strengthen using namespaces in Julia, make method naming less fragile, make baremodule as a default more usable, and make macro based workarounds unnecessary. Maybe it isn’t even possible…

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [May 16, 2018, 5:30pm UTC](https://discourse.julialang.org/t/julep-taking-multiple-dispatch-export-import-binary-compilation-seriously/10882/71 "2018-05-16T17:30:58Z")

</div>

> [@jlperla](#):
>
> Also python, object-oriented matlab, anything with single-dispatch or operator overloading, etc. Basically almost any other language users will come from.

Not really. Arguably, Python evaluates `foo.bar(args...)` in much the same way Julia would evaluate the same expression: evaluate `foo`, then look up the `bar ` attribute of the `foo` object, evaluate the arguments from left to right, then apply the function `foo.bar` to the arguments (internally using `PyObject_Call`). (Of course, in Julia, it’s less common to have the OO style of having objects with function members, except of course for the case where `foo` is a module.) Functions are first-class in Python, as in Julia — e.g. you can assign a method to a variable via `f = foo.bar` and later call `f(args...)`. And of course you can have functions in Python that are not OOP methods, and again `foo(args...)` is evaluated as in Julia: evaluate `foo`, then `args...`, then apply. One big difference is that Python has no concept of a generic function object that dispatches on its arguments: given a callable object `f`, then `f(args...)` in Python will always go to the same code regardless of `args...`. The other main difference is that that in a Python object’s method call `foo.bar(args...)`, the `foo` object is implicitly converted into the first (`self`) argument in the bar definition—essentially, the `foo.bar` method object in Python stores a reference to `foo`—which wouldn’t happen automatically in Julia (though it could be emulated with `getproperty`).

That being said, I totally agree that a conceptual adjustment is required in coming to Julia from OOP languages or other languages centered around different kinds of abstractions.

---

<div class="post-metadata">

**Author:** ![jlperla](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jlperla/32/34332_2.png) [@jlperla](https://discourse.julialang.org/u/jlperla)\
**Post date:** [May 16, 2018, 6:42pm UTC](https://discourse.julialang.org/t/julep-taking-multiple-dispatch-export-import-binary-compilation-seriously/10882/72 "2018-05-16T18:42:36Z")

</div>

> [@stevengj](#):
>
> Not really. Arguably, Python evaluates `foo.bar(args...)` in much the same way Julia would evaluate the same expression: evaluate `foo` , then look up the `bar` attribute of the `foo` object,

I don’t think that is correct, and it is very instructive to see why., which may help understand how to educate people coming into the language

Python looks in the namespace of `foo` to find the `bar` method (i.e. an ADL where only the first argument matters), which Julia does not.

So the basic teaching approach is that you tell people coming from python that all they need to do is turn `foo.bar(args...)` into `bar(foo, args...)` to use multi-dispatch, which is largely correct (until namespaces kick in).

Lets examine how that works with namespaces.  
In python, create a file called `mymod.py` with the text

```julia
class MyClass:
    def bar(self, x):
        return 1

```

Then in a separate file or jupyter, go

```julia
import mymod
foo = mymod.MyClass()
foo.bar(2)

```

This works great, and I have been able to use fully namespace qualified creation for my type (`mymod.MyClass`) But note that there is only one namespace qualification required.

Now, the Julia approach would be to tell people to just convert this by swapping the order. Totally reasonable and easy to teach. So without namespaces it becomes

```julia
foo = MyClass()
bar(foo, 2)

```

But this falls apart with namespaces in Julia. A pythonista told to swap the order with modules has a trickier time

```julia
import mymod
foo = mymod. MyClass() #This part works great in julia
bar(foo, 2) #Fails
mymod.bar(foo, 2) #necessary because the namespace of the foo isn't used

```

This is one of the area of confusion for non-Julia users coming into the language. There are others (e.g. a pythonista might not understand why they can’t create methods of a particular name for a particular type if they use a package).

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [May 16, 2018, 7:07pm UTC](https://discourse.julialang.org/t/julep-taking-multiple-dispatch-export-import-binary-compilation-seriously/10882/73 "2018-05-16T19:07:17Z")

</div>

Here’s my summary of what’s going on in this thread and the other. Take it or leave it. I wanted to write this down so that way people can understand that the responses are not dismissive, but clearly identify the reasons why no one is really gung-ho on implementing this.

## Motivation

It starts with some assumption that Julia’s current function namespacing is hard to understand for people coming from single dispatch languages. Example statements:

> [@jlperla](#):
>
> I have tried to say that **everyone** has been confused on this topic, myself likely the most confused of all through most of it (though I think I finally “get it”). I had suggested from your previous summary of the ADL thread (which talked about multi-dispatch) that you stopped reading and not fully digested the conversation, but I apologize if you had.

There’s no objective data on this, just some people saying “yes I found it confusing” (and others not being confused about it, I’m in the latter camp). It’s normally people who are less familiar with Julia dev saying it’s confusing (which is the where the “contentious split” comes from). This then leads to the conclusion by some that further namespacing of functions to allow for extension in new ways is a feature/design that should be worked towards. However, there’s two objections to that.

## Objection 1: Utility

First of all, there’s the objection about whether it’s actually better. @jeff.bezanson and I have argued that the resulting code from having multiple functions merge on their own could be more non-local and more confusing. For example:

> [@jeff.bezanson](#):
>
> This is closely related to the notion of “dynamic scope” (as opposed to lexical scope), where variables are looked up in calling scopes. A general problem with such approaches is that they make it hard to reason about code. If I write
> 
> ```julia-auto
> f() = sum([1,2,3])
> 
> ```
> 
> in my package, I intend for `f()` to return `6` . In your proposal, unless I misunderstand it, I have no idea what `f` does. If it’s called from a package that defines `sum(x) = 42` , or `a+b = 42` , then `f()` will return `42` . Yes, currently it’s possible to overwrite `Base.sum` or `+` of integers, but we discourage that and print a warning if you do it. But in your proposal it would just be the correct behavior to let another package replace the `sum` or `+` you intended to call.

This is the same thing that I was saying earlier with

> [@ChrisRackauckas](#):
>
> But then what does your code even mean or do? Now your code uses the same syntax as everyone else, but does different things than Base! Now your code adds a `+` function to your number type but then putting that number into `Base.sum` , DiffEq, Optim.jl, NLsolve.jl doesn’t work. Now your “generic functions” don’t work on number types you haven’t specifically programmed them for, so they aren’t generic at all. And if you want to just take over every single dispatch to reprogram someone else’s generic code… isn’t this just Cassette’s contextual dispatch?

So essentially the objection is that the issues you can get from extending functions is really due to abusing existing names and writing something confusing in the first place. I haven’t seen a counter to that in this thread.

## Objection 2: Implementation Feasibility

@stevengj, @yuyichao, @mbauman, etc. have engaged in more technical discussions about implementability of these features. While @jlperla brings up here and in [Function name conflict: ADL / function merging?](https://discourse.julialang.org/t/function-name-conflict-adl-function-merging/10335) that C++ does something similar to what’s proposed, the issue is that there doesn’t seem to be another language that actually achieves this with first-class functions and multiple dispatch. Putting all of those features together is the combination that is difficult to handle. For example:

> [@jeff.bezanson](#):
>
> The key phrase here is “for a given function”. The entire problem is how to identify which function you’re talking about. For example, if a function is just identified by a name (as it is in class-based OO languages) then the rest of your proposal goes through no problem—we could implement it e.g. by having a single namespace of functions.

In Julia, a “function” can be any type with a call added to it. Functions are not distinct from other variables, and names can be dynamically changed since Julia is dynamic. So the implementation has to account for this. So the problem is that @stevengj’s question:

> [@stevengj](#):
>
> For example, think about what happens when a module A defines a new type `T` and extends the iterator functions `Base.start` etcetera to work on T. Then module B calls module A to create a `x = T()` and then calls `Base.sum(x)` , which calls `Base.start(x)` etcetera — this works because `Base.start` has been extended to work on `T` , regardless of where it is called from. How does `Base.sum` call A’s iterator methods if `A.start` and `Base.start` are entirely distinct?

while it’s answered by the simple “it’s easy, C++ does it!” doesn’t necessarily apply to Julia. To keep the nice features of Julia (which are essential to generic programming in this language) and have this kind of merging is completely unknown, and it’s unknown whether it’s feasible. This is why the core devs have proposed technical questions and have asked for PRs / prototypes because it’s unclear how this could even be done. If someone thinks it’s possible, the best way to answer these technical questions is to have a working code that doesn’t have these issues but keeps Julia’s core features in tact.

## Summary

So it’s obviously not making it into Julia v1.0. But it’s also not dismissive to say that it’s not a v2.0 priority either. Julia is not MATLAB/Python/R/C++/etc., it’s different in many ways. The way that Julia does generic programming is relatively simple and dynamic which is quite unique. Trying to bring Julia in line with the other languages should not drop this core element of Julia’s design and usage. Objection 1 is basically noting that these changes don’t necessarily help generic programming in Julia. Objection 2 is noting that coming up with an implementation that doesn’t effect Julia’s generic programming structures is really hard if not impossible. The fact that it’s hard to do and doesn’t have a clear positive outcome is why people aren’t throwing their free time on this project.

## The Evolution of the Julia Community

I think this needs to be ended with a statement on the current state of the Julia community. To outsiders and newcomers, Julia looks new because you just found it. It’s a language where things are being experimented on, it’s a language which is hot on the “what’s new to learn” lists, etc. People jump in thinking there’s huge wholes in the package ecosystem that need to be filled and it’s the wild west. People call it “in beta” since it’s not v1.0.

However, Julia is definitely not in beta and it’s well into its transition into “not new”. There is an established community. There are large state-of-the-art Julia packages/projects (JuMP, DifferentialEquations.jl, Flux.jl, etc.) which make heavy use of Julia’s unique features. There’s common styles, learning materials, etc. There are people hired to work on and with the language. The language itself is gearing up to release its 1.0 long(er)-term support release in just a few months. And for reference, the language was publicly released more than 5 years ago.

Over these last five years, people have built impressive software using the existing language. It’s popularity routinely lists somewhere around things like Haskell and Lisp. There is no question about whether Julia “has made” or will “stick around”: Julia has already made it and has a large enough community to keep it around, not to mention a stronger funding structure and a larger dev community than any of the other open source languages I’ve seen.

This is where the disconnect seems to happen in these kinds of threads. Newcomers tend to make statements like “it will only be adopted if …”, but everyone around here adopted Julia along time ago and is **using Julia not in spite of these differences, but because of these differences.** If you think about the issue like that, it’s probably clear why arguments based on “but I know xyz MATLAB/R/Python people won’t adopt Julia until …” is met with “who cares? Julia is for us because we think it’s better, and we’re willing to share it but don’t want to lose what makes Julia unique”. Yes, Julia isn’t as popular as Python, and we can argue endlessly about how to market better (and whether it’s actually a good thing to be that popular at the cost of being so generic, I’m not entirely convinced that’s a good thing). However, arguments about changing Julia will do much better if it’s not discussed like a shiny new toy to play around with, and instead is discussed as the production-ready software that it is.

This has to mentioned because these kinds of threads have a heavy premise in their motivation that for Julia to be “useful” it may need to adopt xyz from some other language to either have people transition or to implement some feature correctly. But this assumption is wrong. People already have and continue to use/pickup Julia because of what it currently does. You can not argue that the Julia should adopt what everyone else is doing without discussing whether it’s better than what Julia is already doing.

## Me?

As a specific example for this case, DifferentialEquaitons.jl has worked extremely well because of how Julia’s generic programming works. I am not sure how possible this would’ve been in MATLAB/Python/R/C++/etc., at least without substantial amounts of extra work. Maybe there is a way to work out all of this extra namespacing + auto-merging in a way that doesn’t effect the current generic programming structure at all, who knows. But you aren’t going to convince me that Python does generic algorithms better (it doesn’t), and so if you don’t have a way that keeps Julia’s generic functions over abstract number/array types in tact, I don’t want it. If your proposal does a substantial break to this (and other) existing code then it’s not necessarily a good thing unless I have some very tangible benefits. It’s not dismissive to say “if you want the Python way, then go use Python” because they are different for a reason. I have used both and think Julia’s is much better for building exactly the things I have built.

This is long but hopefully puts everyone on the same page. The technical questions and the community statements are heavily intertwined in the discussion, so I hope this helps parse out why those more established seem to be “against” the idea and why this easily raised tensions.

Cheers. ☀

---

<div class="post-metadata">

**Author:** ![jlperla](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jlperla/32/34332_2.png) [@jlperla](https://discourse.julialang.org/u/jlperla)\
**Post date:** [May 16, 2018, 7:20pm UTC](https://discourse.julialang.org/t/julep-taking-multiple-dispatch-export-import-binary-compilation-seriously/10882/74 "2018-05-16T19:20:13Z")

</div>

Much of this I agree with, (though I think your `Objection 1` isn’t how it would be, if such a design was possible) and I think you are leaving out all of the the utility of being able to write practical code with any `using` and without the namespace hacks into `Base` that may be discouraged in theory, but are extremely common, often necessary, and difficult to explain. The likelihood is that an ADL based approach would make generic programming with namespaces much stronger, but that would remain to be proven. At this point it is not clear if **any** language with first-class functions has really cracked the problem of coexisting with namespaces in a way that doesn’t require merging all operators into a single namespace.

But all of that is to say that an ADL based approach is something that should be considered down the road. For now, there have been big communication issues because of confusion on where the real problems lie (if they are fixable and the downsides to fixing them could come later). Documentation, education, and macros could help a lot.

---

<div class="post-metadata">

**Author:** ![ScottPJones](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/scottpjones/32/146_2.png) [@ScottPJones](https://discourse.julialang.org/u/ScottPJones)\
**Post date:** [May 16, 2018, 7:27pm UTC](https://discourse.julialang.org/t/julep-taking-multiple-dispatch-export-import-binary-compilation-seriously/10882/75 "2018-05-16T19:27:19Z")

</div>

> [@jlperla](#):
>
> Documentation, education, and macros could help a lot.

Please, come and review / critique / write some code for what I’m trying to do with the `@api` macro in [APITools.jl](https://github.com/JuliaString/APITools.jl). I know that even in it’s very early, primitive form,  
it’s helped me a lot with the sorts of problems that have been discussed here as I’ve tried to make the `Str` code as generic as possible, keeping the public API & also developer interface easy to extend.

---

<div class="post-metadata">

**Author:** ![ExpandingMan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/expandingman/32/866_2.png) [@ExpandingMan](https://discourse.julialang.org/u/ExpandingMan)\
**Post date:** [May 16, 2018, 7:29pm UTC](https://discourse.julialang.org/t/julep-taking-multiple-dispatch-export-import-binary-compilation-seriously/10882/76 "2018-05-16T19:29:18Z")

</div>

> [@jlperla](#):
>
> I think you are leaving out all of the the utility of being able to write practical code with any `using` and without the namespace hacks into `Base` that may be discouraged in theory, but are extremely common, often necessary, and difficult to explain

This is something that I remain extremely unconvinced of. Julia provides both the `import` and `using` keywords for a reason. If you are absolutely opposed to `using` there is nothing forcing you to using it (pun intended). There is nothing stopping you from extending methods from any other package just as you would for `Base`, as has been said in the past the only thing that makes `Base` special is that every module has an implicit `using Base`. I rarely encounter anything I would classify as a “[weird] namespace hack into `Base`”, certainly not in any of the core ecosystems I’m familiar with.

If there’s a problem here I’ve never run into it.

---

<div class="post-metadata">

**Author:** ![mbauman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mbauman/32/31082_2.png) [@mbauman](https://discourse.julialang.org/u/mbauman)\
**Post date:** [May 16, 2018, 7:38pm UTC](https://discourse.julialang.org/t/julep-taking-multiple-dispatch-export-import-binary-compilation-seriously/10882/77 "2018-05-16T19:38:44Z")

</div>

> [@jlperla](#):
>
> At this point it is not clear if **any** language with first-class functions has really cracked the problem of coexisting with namespaces in a way that doesn’t require merging all operators into a single namespace.

Sure there is, its name is Julia. We don’t require merging functions, and jamming puns into Base functions really isn’t all that common. Infix operators are indeed tricky, as they aren’t namespace-able, but that’s one of the reasons we have _so many_ unicode infix operators — to allow folks to write compact syntax that _doesn’t_ have difficulties with generic programming.

With ADL, it would _simply not be possible_ to write a cromulent function that uses a function named `fit!` with our existing ecosystem unless you explicitly enumerate all the argument types that support your meaning of `fit!`. Ref this comment: [Function name conflict: ADL / function merging? - #92 by mbauman](https://discourse.julialang.org/t/function-name-conflict-adl-function-merging/10335/92)

Now, I agree there’s a pain point when two modules export the same (un-merged) name. That’s annoying. That’s why my concrete and simple suggestion above is a tool (editor tool, ideally) that allows you to automatically flush out the names to be explicitly used from a `using: …`. And there’s a pain point in having a place where package developers can agree on common names to extend, but that discussion needs to happen somewhere for downstream users to make sense of it all.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [May 16, 2018, 7:41pm UTC](https://discourse.julialang.org/t/julep-taking-multiple-dispatch-export-import-binary-compilation-seriously/10882/78 "2018-05-16T19:41:35Z")

</div>

> [@jlperla](#):
>
> I think you are leaving out all of the the utility of being able to write practical code with any `using` and without the namespace hacks into `Base` that may be discouraged in theory, but are extremely common, often necessary, and difficult to explain.

Extending functions isn’t a hack, it’s easily expressible in the language with its basic constructs. It’s not “discouraged in theory”, it’s directly recommended in the documentation and the associated interfaces. And I don’t think it’s difficult to explain: variables in Julia are in namespaces, and adding dispatches to functions is a pretty core concept of the multiple dispatch structure.

What is not recommended is extending functions on Base types (type-piracy). That’s where Objection 1 comes in. What some people have said is that the problem isn’t language design related, the problem is code design. No matter how you can write it, the fact that you are making some basic name or function act a way that is not how it usually acts will always be confusing. It will always be confusing for a `sum` function to not actually take a `sum` (and if it did just take a `sum`, just extend `Base.sum` and we have no arguments since any function merging will act exactly the same!). Whether there’s special syntax to allow people to write other `sum` functions, or whether it’s explicitly discouraged as type-piracy, doesn’t give me a good reason to want a code to ever do this in the first place. If it’s really different, use a different name (or a name in a different namespace), in which case this is a non-issue.

I am thinking about it like whitespace sensitivity in Python. Python requires that you indent your code correctly. There are ways to engineer a language/parser so that’s not required (ex: `end` like in Julia). But at the end of the day, everyone should indent their code correctly so whichever way you choose is a non-issue or at least a very surface-level issue.

Here we have a coding style choice which isn’t necessarily a clear way of writing code. Even if there is a good way to namespace a function so that way you can have the same name do two things and but have it automatically merge in many cases so that generic programming isn’t broken, you shouldn’t actually use it. If the function is actually doing something different, give it a different name. Or stick a macro on it so it’s clear that this line changes the behavior of the functions. While we have so far had this “accidentally” (?) enforced via how extension works, I don’t see a huge benefit for getting rid of this enforcement. So while I see why this can cause confusion, I respectfully disagree that this is a language problem and not a code problem.

---

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [May 16, 2018, 7:53pm UTC](https://discourse.julialang.org/t/julep-taking-multiple-dispatch-export-import-binary-compilation-seriously/10882/79 "2018-05-16T19:53:31Z")

</div>

> [@mbauman](#):
>
> Now, I agree there’s a pain point when two modules export the same (un-merged) name. That’s annoying. That’s why my concrete and simple suggestion above is a tool (editor tool, ideally) that allows you to

At one point I wished for something like this to be available:

```julia
using AModule except thisfun andthatfun

```

where `except` would be a keyword. It would accept the export of all symbols from `AModule` except `thisfun` and `andthatfun`.

EDIT: THE except IS NOT NEEDED IN FACT. SEE STEFAN’S POST BELOW THAT EXPLAINS HOW AN IMPORT IS ALL IT TAKES.

---

<div class="post-metadata">

**Author:** ![jlperla](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jlperla/32/34332_2.png) [@jlperla](https://discourse.julialang.org/u/jlperla)\
**Post date:** [May 16, 2018, 8:03pm UTC](https://discourse.julialang.org/t/julep-taking-multiple-dispatch-export-import-binary-compilation-seriously/10882/80 "2018-05-16T20:03:15Z")

</div>

> [@ChrisRackauckas](#):
>
> If the function is actually doing something different, give it a different name.

That is only because namespaces right now are used (loosely) to represent concepts, where they are more formally defined in other languages.

There is no reason that there can only be a single active “solve” concept at any point in time operating on a dijoint subset of classes. Being more formal in making sure that these things can never clash is what Concepts/Typeclasses/etc. can do, but that is a Julia v5.0 (if ever). Julia’s namespaces do not do much to help, in practice, with isolating concepts because any time you want to use operators you end up having to agree on a namespace (usually `Base`).

So you get to use `solve` in differential equations and the next package that comes around and just wanted to use the word “solve” has to pick a different function? What about operators used in base?

[Previous page](https://discourse.julialang.org/t/julep-taking-multiple-dispatch-export-import-binary-compilation-seriously/10882.md?page=3)

[Next page](https://discourse.julialang.org/t/julep-taking-multiple-dispatch-export-import-binary-compilation-seriously/10882.md?page=5)