Named parameters with type overloads

General Usage
1

Hi,

I’m relatively new to Julia. Here’s a thing that puzzles me. If you have a simple overloaded function like this:

function a( x :: Float64 )
         print( "Float version" )
end
function a( x :: Integer )
         print( "Integer version" )
end

it works exactly as you’d expect. But if you have named parameters, and the name is the same but the types differ, as in:

function b( ; x :: Float64 )
         print( "Float version" )
end

function b( ; x :: Integer )
         print( "Integer version" )
end

Only the last declared function works; so calling b here with an integer works
as I’d expect, but with a float type you get:

   b(x=10.0)

   ERROR: TypeError: in #B, in typeassert, expected Integer, got Float64

Is this a bug, or deliberate?

Tested with Julia 1.03.

Graham

1 Like
2

Yeah, this is intentional. From the manual section on methods:

Keyword arguments behave quite differently from ordinary positional arguments. In particular, they do not participate in method dispatch. Methods are dispatched based only on positional arguments, with keyword arguments processed after the matching method is identified.

So, in other words, you can’t have two different methods whose signatures differ only in the types of their keyword arguments. You can, however, do:

function b(; x)
  b(x)
end

function b(x::Integer)
  println("Integer")
end

function b(x::Float64)
  println("Float")
end

julia> b(x=1)
Integer

julia> b(x=1.0)
Float
9 Likes
3

Thanks for that. I should have read the docs more carefully.

I like your code - I wouldn’t have thought of that.

What’s the reasoning behind this? In other languages I’m aware of that have this feature, named parameters are there to help document code and cut down on errors in call lines but can’t change the meaning of the code. Having the code behave differently seems counter-intuitive to me.

4

In the example above, the it is not the same code that behaves differently, but different code is called depending on the type. You may want to read

https://docs.julialang.org/en/v1/manual/methods/

5

If I have:

function b( x :: Float64 )
         print( "Float version" )
end

and I add a semicolon:

function b( ; x :: Float64 )
         print( "Float version" )
end

I end up with a function that may never be called at all. I find that very odd. We’ve established that this is intentional, but I don’t understand what is the thinking behind it is.

6

Sorry, I am not getting what the problem is: whether the function gets called depends on the code using it. Eg

b()

would call it.

Can you clarify what the “it” refers to in the sentence? Multiple dispatch? Or how keyword arguments behave?

Did you read the section I linked?

7

keyword arguments. In the world I come from (Ada), you’d use keyword arguments when calling a function to improve documentation, and to reduce the chance of mismatching call-lines. That was what I wanted and expected to happen here.

8

That’s their use in Julia, too. Also, as mentioned above, keyword args don’t participate in method dispatch. What happens in your original example is the following:

function b( ; x :: Float64 )
    ...
end

defined a method for b that is called for all b(; ...) forms, ie no positional arguments. It needs to have an x keyword argument, and it needs to be a Float64, but this does not affect dispatch.

Then you define

function b( ; x :: Integer )
         print( "Integer version" )
end

which effectively replaces the previous method (since, remember, keyword arguments don’t affect dispatch), with the same constraints, except now it expects an <: Integer value. Which is why you get the error.

1 Like
9

Tamas,

I’m aware that

and rdeits post above demonstrates that this is intentional. I think we all agree on this. My question is: why? What is the language designers reasoning here?

1 Like
10

I can only guess, but I would say it is simply tradition: CLOS doesn’t dispatch on keyword arguments either. I can imagine an alternative design where dispatch happens on keyword arguments, too. Do you have a specific reason for preferring it?

11

I think it is purely historical reasoning at this point.
Up until julia 0.7 kwargs didn’t participate in specialisation (let alone dispatch).
For, I believe, implementation reasons. (The switch of kwargs came with the NamedTuple type)

That had a fair bit of momentum towards not using kwargs for anything really important.

I would not be surprised if julia 2.0 had dispatch on kwarg types

2 Likes
12

I thought it had to do with potential ambiguity,

foo(a;b::Int=0,c)="Int Version"
foo(a;b::Float64=0.0,c)="Float Version"

Now for foo(1;c=2.0) which methods do you use?

EDIT: Forgot a default for b.

1 Like
13

hmmm, I don’t think so. (but could be wrong)

I’m not sure if there is any more ambiguity there than for

julia> bar(a, b::Int=1)="Int Ver"
bar (generic function with 2 methods)

julia> bar(a, b::Float64=1.0)="Float ver"
bar (generic function with 3 methods)

julia> bar(1)
"Float ver"

That resolution occurs become the method bar(::Any) is replaced.

More generally, we deal with ambiguities already.
e.g.

julia> buzz(::Any, ::Int)="Int Ver"
buzz (generic function with 1 method)

julia> buzz(::Float64, ::Any)="Float Ver"
buzz (generic function with 2 methods)

julia> buzz(1.0, 1)
ERROR: MethodError: buzz(::Float64, ::Int64) is ambiguous. Candidates:
  buzz(::Float64, ::Any) in Main at REPL[5]:1
  buzz(::Any, ::Int64) in Main at REPL[4]:1
Possible fix, define
  buzz(::Float64, ::Int64)
Stacktrace:
 [1] top-level scope at none:0
14

Jeff and I discussed this for a long time before he implemented it and very much considered it as a clean slate design problem. If the current behavior matches CLOS, I think that’s just convergent evolution, since as far as I’m aware, neither of us have actively used CLOS. @jeff.bezanson can correct me if he was influenced by CLOS more than I realize.

The basic problem is this: in the presence of arbitrary optional named keyword arguments and multiple dispatch, how in the world does that work? I’d be interested in hearing proposals because we couldn’t come up with anything at the time that was understandable by mere mortals or implementable efficiently which combined the two.

4 Likes
15

FWIW, I think that not dispatching on keywords is the sensible solution: even if the semantics & implementation are resolved, it can lead to quite complicated behavior. Which sometimes does make sense, but then one can have dispatch implemented by a kernel with positional arguments.

BTW, Julia is already more flexible than CLOS in that it does not require the same positional arguments for methods of a function.

1 Like
16

I don’t think I’ve ever heard a CLOS user admit that there’s something it can’t do before :joy_cat:. But I’m sure there’s an extension that allows it!

2 Likes
17

Keep in mind that I am a Julia user now :wink: I am not an expert at the Metaobject Protocol, but I think that congruency is a requirement for CLOS.

I find it interesting that I always thought of this requirement as something that could not be otherwise, then when I started Julia I saw that the restriction is kind if arbitrary. Not sticking to it allows so many nice idioms, eg optional arguments in front like rand([rng], ...). Maybe dispatch on kwargs could have similar benefits, but I have to admit that I don’t quite see them at the moment.

1 Like
18

More useful to me at least would be dispatching on which kwargs are present.
For example the sort family with vs without it’s by kwarg,
and the reduce family with vs without it’s dims kwarg.

I assume that the one would need to dispatch on both name and type is what makes it complex.
This is contrasted to positional where one dispatches on position and type.
And name is unordered vs position being ordered.
Though I’m not yet seeing how that complexity would confuse users or be slow.
(Possibly relates to kwsort which I’ve never really gronked the purpose of)

19

It seems like having multiple dispatch on keyword arguments would expand the power of the language and would also have the benefit of consistency in behavior. Much like the original poster, I also expected multiple dispatch to work on keyword arguments based on my experience with positional arguments. I am interested in understanding the difficulty of using multiple dispatch on keyword arguments. Here are some of my thoughts:

At least from my naive point of view as a user, it seems like positional arguments would work as usual, keyword arguments would match name and type, and the optional kwargs could work similar to varargs where types could be declared optionally. Consider the following example,

function foo(a,b;c::Int64,d::Int64,kwargs...)
    #...
end

function foo(a,b;c::Float64,d::Float64,kwargs...)
    #...
end

function foo(a,b;c::Float64,d::Float64,e::Int64,kwargs...)
    #...
end

Each function is distinguished by the keyword types. As far as I can tell, the most important constraints are imposing unique argument names and distinguishable types in order to avoid method ambiguity, which does not introduce a new rule. In the example above, foo(1,2;c=.3,d=.2,e=.3) would call the second version of foo because it is not an Int64. However, foo(1,2;c=.3,d=.2,e=3) would call the third version of foo because it is an Int64. The rule in the later case is to call the most specific matching method. For a lack of better word, let’s call this the rule of specificity, which seems to already be in place, e.g. myfun(a::Int64,b::Int64) and myfun(a,b) are not ambiguous.

One possible conflict may arise when kwargs… is contained by a type. For example, replace the second version of foo above with the following:

function foo(a,b;c::Float64,d::Float64,kwargs::Int64...)
    #...
end

Let’s call this 2’. Which method should be called when given: foo(1,2;c=.3,d=.2,e=3)? In this case, 2’ and 3 both match in terms of type and name because e=3 could be in kwargs.. . So one solution might be to prohibit type declarations on kwargs... . However, I don’t think that is necessary. In keeping with the rule of specificity, dispatch on version 3 because the keyword e::Int64 is explicitly declared in the method definition, making it more specific than 2’.

That seems straightforward to me as a user, especially with concrete examples illustrating how the rules work. Considering that language developers struggled with this, I suspect I am missing something important. Is there an edge case that I failed to consider? Is there a large cost in parsing unordered keyword arguments and enforcing these rules? I would be interested in understanding more about this.

20

It looks very complicated to me, with exceptions and multiple layers.

But if you find this useful, you can already call an internal function and implement dispatch using that, eg

foo(a, b; c = nothing, d = nothing, e = nothing) = _foo(a, b, c, d, e)
_foo(a, b, c::Int64, d::Int64, ::Nothing) = ...

Alternatively, if you dislike nothing as a sentinel, you can collect the kwargs into a NamedTuple and dispatch on the type of that (it includes the variable names).

I was not aware of dispatch for kwargs being on the table, can you link the discussion?

2 Likes