Julep: Taking multiple dispatch,export,import,binary compilation seriously

[StefanKarpinski]

@jlperla

In single-dispatch languages, the basic lookup rule is that whatever the namespace of the type you use as the first argument determines the relevant set of methods for a given function. The natural generalization of this is that with multi-dispatch, is that the namespaces of the types you have as the arguments determines the relevant set of methods for a given function. This is exactly what ADL does.

I think this is probably the single clearest explanation of the ADL technical position so far and I wanted to highlight it as such. The explanation misses one fairly important aspect of the issue, however. Unlike single dispatch in C++ and other object-oriented languages, which by definition is dynamic, ADL is strictly a static name disambiguation done at compile time. In other words, in C++ there is always an equivalent program you could have written without ADL using fully qualified function names.¹ This is not a trivial difference and it means that suggesting that ADL is the natural extension of single dispatch to multiple arguments is highly misleading, if not flat out incorrect. Doing dynamic name resolution and dynamic dispatch on all of the arguments is not at all like C++’s ADL… it’s not like anything in any language that exists, which frankly gives me a great deal of pause. Julia already has one of the most powerful dynamic dispatch mechanisms anywhere; making it more complicated seems to pose a real danger of making it impossible to reason about and unusable for anything beyond toy programs.

¹ After template expansion: each expansion of templates can lead to ADL resolving a different function to call. This is still all static and done entirely at compile time, however.

@stevengj

It may be interesting to think about as an academic exercise, but I think you’ve confused people understanding how to better explain the difference between Julia’s dispatch and the ADL you are used to with agreeing that there is something wrong with Julia’s approach (which is common to all all other dynamic multiple-dispatch languages AFAIK) that we need to plan to “fix” in Julia 2.0.

I want to second this. I’m very glad to have understood the ADL perspective better. However, I’d like to clarify a few points:

1. It was not new to us that single dispatch has a nice property in terms of avoiding putting method names into global namespaces. I’ve had dozens of conversations over many years with various core devs (including but not limited to Jeff) about this issue and various ways we could get the benefits of single dispatch for namespacing without messing up what is great about Julia’s multiple dispatch and generic function system. So far nothing has been a clear win.

2. Technical problems are not the only thing preventing us from adopting ADL. ADL seems to be somewhat controversial even in C++ even though it is just syntactic sugar to avoid having to write out fully qualified function names. In Julia, you cannot statically resolve ADL because there are no static types by which to do so. Even if we came up with a dynamic analogue of ADL that did both dynamic argument-dependent function lookup and dynamic dispatch, I’m very far from convinced that it would be a good idea. At the risk of speaking for Jeff, I’m pretty sure he feels similarly.

3. The goal of programming “without using” is not universal. In fact, I think the only person who has really enthusiastically espoused doing away with using is @jlperla. I understand why it’s appealing—less boilerplate is always nice—but I find the notion that you have no idea where methods you call might be coming from pretty unsettling. It works in C++ precisely because ADL is just a static rewrite to an equivalent fully qualified form. That fundamentally cannot be done in Julia since there are no static types. If we had “dynamic ADL without using” then the expression f(...) could, in general, call literally any method definition for the function name f in any code anywhere, even in code that I don’t use (even indirectly) and don’t know anything about. That is some seriously spooky action at a distance. In C++ I had to at least mention a type in my code that belongs to the namespace where the function definition lives. In Julia, there can be literally no connection between the caller and the method that gets called.

@jlperla

ADL is the only hope for something that would extend the current semantics and let people better segment namespaces of methods (essentially by have no namespaces for functions).

ADL is absolutely not the only approach to this issue and the fully unlimited “using-free” form of ADL that you support is definitely not the only option. There are several other ways to address this issue that have been discussed for Julia in the past. One of them is making the syntax x.f(y) mean something like moduleof(x).f(x, y). Another one is to provide better support for explicit generic function merging. Another would be automatic function merging when different functions with the same name are brought into scope by using. Yet another is deciding that all of the cures are worse than the malady. It’s a one-time annoyance to import the function you want to call, but it’s hardly the usability disaster that some have tried to make it out to be. Every paradigm has it’s strengths and weaknesses. Perhaps managing global names is just one of the things that’s a bit annoying in multiple dispatch.