I'm confused about methods

General Usage
1

This surprises me (particularly the part where f() evaluates to 1):

julia> begin
           f(a)=2
           f(a,b)=3
           f(;a,b)=4
           f()=1
       end
f (generic function with 3 methods)

julia> methods(f)
# 3 methods for generic function "f":
[1] f(; a, b) in Main at REPL[1]:5
[2] f(a) in Main at REPL[1]:2
[3] f(a, b) in Main at REPL[1]:3

julia> f()
1

julia> f(1)
2

julia> f(1,2)
3

julia> f(a=1,b=2)
4

julia>

For fun, repeat with the f()=1 moved to the top of the list.

1 Like
2

Why does it surprise you? What did you expect f() to give and why?

1 Like
3

Notice that f() doesn’t appear in the return from methods(). On the other hand, if I never define f(; a,b) then methods(f) gives

# 3 methods for generic function "f":
[1] f() in Main at REPL[1]:1
[2] f(a) in Main at REPL[2]:1
[3] f(a, b) in Main at REPL[3]:1

Also, if I define the four methods in this order:

begin
	f()=1
	f(a)=2
	f(a,b)=3
	f(;a,b)=4
end

and run f() I get an Error: `UndefKeywordError: keyword argument a not assigned.

So I wasn’t sure that it was acceptable to have both an emtpy arg list method f() and a purely keyword-based method f(; a,b) defined at the same time.

So my surprise is that I get different behaviors depending on the order of the definitions and that in the original posted session, methods(f) listed 3 methods, but I could demonstrate 4.

3 Likes
4 
❯ julia --warn-overwrite=yes -q
julia> begin
           f()=1
           f(a)=2
           f(a,b)=3
           f(;a,b)=4
       end
WARNING: Method definition f() in module Main at REPL[1]:2 overwritten at REPL[1]:5.
f (generic function with 3 methods)
4 Likes
5

Keyword arguments don’t participate in dispatch, so it is expected that f() = 1 completely overrides the previous definition f(; a, b) = 4 and that’s why there are only 3 methods listed. It also explains why f() works in your first example and not in your second example.

It looks like you actually stumbled upon a Julia bug here when printing the methodtable. The first method should be displayed as f(), not f(; a, b). Would you mind opening an issue on GitHub about this?

Edit: Wait, in this example f(a=1,b=2) shouldn’t work, so this seems like an issue of the method overriding not working like it should, not just a printing bug.

4 Likes
6

I feel like I should win a prize if I’ve found a bug this simple!

Anyway, simplifying further:

Scenario 1:

C:\Users\klaff>julia -q
julia> f()=1
f (generic function with 1 method)

julia> f(;a)=2
f (generic function with 1 method)

julia> methods(f)
# 1 method for generic function "f":
[1] f(; a) in Main at REPL[2]:1

julia> f()
ERROR: UndefKeywordError: keyword argument a not assigned
Stacktrace:
 [1] f() at .\REPL[2]:1
 [2] top-level scope at REPL[4]:1

julia> f(a=1)
2

julia>

Scenario 2:

C:\Users\klaff>julia -q
julia> f(;a)=2
f (generic function with 1 method)

julia> f()=1
f (generic function with 1 method)

julia> methods(f)
# 1 method for generic function "f":
[1] f(; a) in Main at REPL[2]:1

julia> f()
1

julia> f(a=1)
2

julia>

I’m happy to make an issue although I’m not sure exactly what it is. I don’t think order of definition should affect whether a method gets overwritten (but it should affect which is left standing). Certainly in scenario 2 the return from methods(f) is non consistent with reality (both methods are reachable).

How do I know (from a doc standpoint) whether those two methods should be allowed to coexist?

EDIT: If you change the above to f(;a=1)=2, then there are no surprises Whichever is defined last is the one method you get.

7

This is not a bug but intended behavior of how function definitions with keyword arguments work internally (refer to the developer documentation for more information). Long story short, f(; a) = 2 actually defines a f() method without any keyword arguments. If defined after f() = 1, this effectively overwrites the original definition of f(), and so call to f() now errors and asks for keyword argument a. In the second scenario, the f() = 1 instead overwrites the f() method defined by f(; a) = 2, which is why f() will return 1.

1 Like
8

They should never coexist at the same time, because they have exactly the same signature. (Keyword arguments aren’t actually part of a method’s signature.)

In scenario 1, everything is working as it should; f()=1 gets overridden by f(;a)=2, so that’s exactly the behavior you observe.

In scenario 2, f(;a)=2 should therefore also be completely overridden by f()=1, so methods(f) should only print f(), not f(; a) and f(a=1) should therefore be an UndefVarError.

1 Like
9

The way function definitions are lowered should only be an implementation detail though, this seems more like an unintended side effect of that, instead of a feature. From a user perspective, this behavior really doesn’t make a lot of sense, at least to me.

2 Likes
10

I think you are correct, and this text from Methods · The Julia Language seems most relevant:

Keyword arguments behave quite differently from ordinary positional arguments. In particular, they do not participate in method dispatch. Methods are dispatched based only on positional arguments, with keyword arguments processed after the matching method is identified.

I’ll submit an issue against scenario 2.

1 Like
11

Even if f() is overwritten in scenario 2, the keyword sorter and the compiler-generated function remains and so f(; a) would still work. I would say this is pretty intuitive behavior for me, since although the method signature is the same, method with keyword arguments is different from one without.

12

This still seems a bit like a potential pitfall to me, since it’s not at all clear form looking at methods(f) and it seems weird that defining them in different orders either overwrites the previous method of f completely or only partially.

2 Likes
13

I don’t think it can be correct that in scenario 2 there are two different return values. That’s inconsistent with f() being overwritten.

2 Likes
14

Calling f(a=1) doesn’t call f() but something along the line of #f#1(a), which was defined with f(; a). The f() definition only handles the case where no keyword is passed (default keyword argument value, for example).

15

Well in both cases the keyword sorter and the generated positional argument-only method still exist. The only method overridden is the call without any specified keyword, so I wouldn’t say the method was overwritten completely.

16

I entered an issue https://github.com/JuliaLang/julia/issues/38066.

I’m a bit out of my depth at this point and will let wiser folks sort it out.

1 Like
17

But AFAIK, it’s not callable in the current worldage anymore, so from a regular user’s perspective, I don’t see how that’s any different than the method being completely overridden.

18 
C:\Users\klaffedk>julia -q
julia> f(;a)=2
f (generic function with 1 method)

julia> f()=1
f (generic function with 1 method)

julia> @code_llvm f()

;  @ REPL[2]:1 within `f'
; Function Attrs: uwtable
define i64 @julia_f_106() #0 {
top:
  ret i64 1
}

julia> @code_llvm f(a=1)

;  @ REPL[1]:1 within `f##kw'
; Function Attrs: uwtable
define i64 @"julia_f##kw_133"([1 x i64]* nocapture nonnull readonly dereferenceable(8)) #0 {
top:
  ret i64 2
}

I can call both methods here.

19

Sorry, I meant only in the first scenario

1 Like
20

Perhaps the current behavior is not as intuitive, but it is sometimes useful to provide a default behavior for methods with keyword arguments. Maybe a warning in the REPL would help?