What is the motivation for Julia not dispatching on keyword arguments?

I have come to peace with the fact Julia does not dispatch on keyword arguments, but I’m still unsure as to why this is the case.

In this StackOverflow thread, a commenter suggests that keyword arguments do not participate in dispatch because this would lead to a combinatorial explosion in the number of methods generated.

I might be misunderstanding here, but to me, that seems like an argument for not specialising on keyword arguments, rather than not dispatching.

I would have assumed that the number of methods is determined by how many the user writes.

Any thoughts on this topic would be appreciated.

2 Likes

I don’t know the answer to your question, but I discovered (by accident when my code behaved unexpectedly) that Julia does indeed sometimes dispatch on keyword arguments. See: Keyword arguments affect methods dispatch · Issue #9498 · JuliaLang/julia · GitHub

1 Like

I suspect that the main difficulty is that ordering method specificity becomes impossible in many cases with keyword arguments, so you would end up with lots more MethodError: ambiguous cases. The same issue doesn’t arise as often with positional arguments because you tend not to have large numbers of positional arguments in multi-method functions (if there are lots of parameters, you use keywords instead).

See also the discussion in keyword arguments · Issue #485 · JuliaLang/julia · GitHub, where this feature was hashed out. @StefanKarpinski wrote:

The main consideration here is making keyword arguments not completely impossibly complicated to understand in the presence of multiple dispatch.

6 Likes

Compare to the existing optional positional arguments and you’ll see that specialization isn’t involved yet, it’s entirely about multimethods with respect to arity:

(This isn't code, I just need formatting) 
foo(a=1,b=2) = ...
generates
foo() = foo(1, 2)
foo(a) = foo(a, 2)
foo(a,b) = ...

n optional arguments make 1+n methods instead of just the original foo(a=1,b=2) because they need to overwrite previous (or be overwritten by subsequent) foo() and foo(a) methods to prevent ambiguous dispatch of calls.
Contrast that with hypothetical optional keyword arguments:

foo(;a=1,b=2) = ...
generates
foo() = foo(;a=1,b=2)
foo(;a) = foo(;a=a, b=2)
foo(;b) = foo(;a=1, b=b)
foo(;a, b) = ...

This short example doesn’t really show off the sum of combinations for space, but n optional keyword arguments would make \sum_{k=0}^{n} C(n, k)=2^n methods (2^n grows slower than the (1+n)! in the linked comment, see binomial theorem to prove that equality). That gets real big real fast, and the only way to avoid making so many methods (to store and to compile for) is to not make multiple methods with respect to keyword arguments.

That’s just one of the implementation problems, but even if there weren’t any, I wouldn’t even want to dispatch over keyword arguments. Despite the earlier optional positional arguments example, it’s far more typical that a foo(a) method does not forward to foo(a,b), but does something entirely different, e.g. -x vs x-y. On the other hand, keyword arguments basically exist to override a few select values in a large set of defaults, and that’s the only way writing arguments out of order has any convenience e.g. plot(x, y; color="red") does the same thing as plot(x, y; linestyle="dots"). Performance would still need specialization, and that does happen; reflection is harder but the underlying kwcall sorts the arguments into positional arguments and specializes over them.

6 Likes

Ah, I hadn’t clicked that multiple methods were generated in this situation. Thank you for clarifying that!