# Unexpected sensitivity to order of method definitions

**URL:** <https://discourse.julialang.org/t/unexpected-sensitivity-to-order-of-method-definitions/106510>\
**Category:** General Usage\
**Created:** [November 21, 2023, 7:51am UTC](https://discourse.julialang.org/t/unexpected-sensitivity-to-order-of-method-definitions/106510 "2023-11-21T07:51:10Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![tverho](https://avatars.discourse-cdn.com/v4/letter/t/839c29/32.png) [@tverho](https://discourse.julialang.org/u/tverho)\
**Post date:** [November 21, 2023, 7:51am UTC](https://discourse.julialang.org/t/unexpected-sensitivity-to-order-of-method-definitions/106510/1 "2023-11-21T07:51:11Z")

</div>

The following works like a charm:

```julia
function f(a; b)
    a + b
end

function f(a)
    a + a
end

julia> f(2)
4
julia> f(2,b=1)
3

```

But reversing the order results in an error:

```julia
function f(a)
    a + a
end

function f(a; b)
    a + b
end

julia> f(2)
ERROR: UndefKeywordError: keyword argument `b` not assigned

```

This seems like bug but maybe I’m missing something? I’m running Julia 1.9.1.

---

<div class="post-metadata">

**Author:** ![Asmund\_Hjulstad](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/asmund_hjulstad/32/31820_2.png) [@Asmund\_Hjulstad](https://discourse.julialang.org/u/Asmund_Hjulstad)\
**Post date:** [November 21, 2023, 9:02am UTC](https://discourse.julialang.org/t/unexpected-sensitivity-to-order-of-method-definitions/106510/2 "2023-11-21T09:02:04Z")

</div>

The definitions override each other, I think. `methods(f)` shows the list of methods that have been defined.

---

<div class="post-metadata">

**Author:** ![jling](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jling/32/212909_2.png) [@jling](https://discourse.julialang.org/u/jling)\
**Post date:** [November 21, 2023, 9:31am UTC](https://discourse.julialang.org/t/unexpected-sensitivity-to-order-of-method-definitions/106510/3 "2023-11-21T09:31:59Z")

</div>

Keyword arguments don’t participate in dispatch

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [November 21, 2023, 10:30am UTC](https://discourse.julialang.org/t/unexpected-sensitivity-to-order-of-method-definitions/106510/4 "2023-11-21T10:30:31Z")

</div>

There’s a hint in the method count displayed when you define the functions:

```julia
julia> f(a) = a + a
f (generic function with 1 method)

julia> f(a; b) = a + b
f (generic function with 1 method)

```

As you see there’s only 1 method for `f` after the second definition, which tells you that the first definition was overwritten.

---

<div class="post-metadata">

**Author:** ![tverho](https://avatars.discourse-cdn.com/v4/letter/t/839c29/32.png) [@tverho](https://discourse.julialang.org/u/tverho)\
**Post date:** [November 21, 2023, 10:38am UTC](https://discourse.julialang.org/t/unexpected-sensitivity-to-order-of-method-definitions/106510/5 "2023-11-21T10:38:23Z")

</div>

Ok but somehow the first version still works, the first definition doesn’t actually get overwritten even though `methods(f)` says there’s only 1 method…?

---

<div class="post-metadata">

**Author:** ![nsajko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nsajko/32/221187_2.png) [@nsajko](https://discourse.julialang.org/u/nsajko)\
**Post date:** [November 21, 2023, 11:12am UTC](https://discourse.julialang.org/t/unexpected-sensitivity-to-order-of-method-definitions/106510/6 "2023-11-21T11:12:09Z")

</div>

> [@tverho](#):
>
> the first definition doesn’t actually get overwritten

Not sure what you mean.

Anyway, you can start Julia with `--warn-overwrite=yes` to make it warn on method overwrite. I always start it like that.

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [November 21, 2023, 11:15am UTC](https://discourse.julialang.org/t/unexpected-sensitivity-to-order-of-method-definitions/106510/7 "2023-11-21T11:15:54Z")

</div>

I think Tuuka’s point is that:

```julia
C:\>julia -q --warn-overwrite=yes

julia> f(a; b) = a + b
f (generic function with 1 method)

julia> f(a) = a + a
WARNING: Method definition f(Any) in module Main at REPL[1]:1 overwritten at REPL[2]:1.
f (generic function with 1 method)

julia> f(2, b = 1)
3

```

this shouldn’t work if `f(a, b = 1)` really got overwritten. I think this is:

> <https://github.com/JuliaLang/julia/issues/42207>
>
> This is also shown in issue \[#9498\](https://github.com/JuliaLang/julia/issues/94…98#issuecomment-504363878), but that issue is set for 2.0. The issue here is for possible intermediate solutions to prevent bug introduction in codes caused by re-ordering of function definitions. 
> 
> \`\`\`julia
> f1(a,b)="no extra";f1(a,b;kwargs...) = ";kwargs..."; (f1(1,1), f1(1,1,k=1))
> \`\`\`
> gives \`(";kwargs...", ";kwargs...")\`. And \`methods(f1)\` gives 1 method. So this is consistent with the last definition overwriting the first. However,
> \`\`\`julia
> f2(a,b;kwargs...) = ";kwargs..."; f2(a,b)="no extra"; (f2(1,1), f2(1,1,k=1))
> \`\`\`
> gives \`("no extra", ";kwargs...")\`. But \`methods(f2)\` still gives 1 method. Yet, it does not seem like the last definition overwrote the first. So I see two problems:
> 1) The output of \`methods\` seems wrong?
> 2) It seems unstable that overwriting only occurs when the kwargs version comes last, but not the other way around?
>    
> As I understood from #9498, these are not possible to solve without introducing breaking changes and will not be solved in 1.x? 
> However, it would still be nice to (at least untill #9498 is solved) have some warning or similar to highlight that these patterns might easily introduce bugs. 
> 
> (Tested on 1.6.2 and 1.7.0-beta4)

which was filed over two years ago but didn’t get any responses.

---

<div class="post-metadata">

**Author:** ![Asmund\_Hjulstad](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/asmund_hjulstad/32/31820_2.png) [@Asmund\_Hjulstad](https://discourse.julialang.org/u/Asmund_Hjulstad)\
**Post date:** [November 21, 2023, 12:12pm UTC](https://discourse.julialang.org/t/unexpected-sensitivity-to-order-of-method-definitions/106510/8 "2023-11-21T12:12:58Z")

</div>

Indeed. It doesn’t get overwritten, they are both there… Confusing.

```julia
julia> f(a; b) = a + b
f (generic function with 1 method)

julia> f(a) = a + a
f (generic function with 1 method)

julia> f(2, b = 1)
3

julia> methods(f)
# 1 method for generic function "f" from Main:
 [1] f(a; b)
     @ REPL[2]:1

julia> f(a) = 10a
f (generic function with 1 method)

julia> f(20)
200

julia> f(20, b=1)
21

```

---

<div class="post-metadata">

**Author:** ![Benny](https://avatars.discourse-cdn.com/v4/letter/b/49beb7/32.png) [@Benny](https://discourse.julialang.org/u/Benny)\
**Post date:** [November 21, 2023, 2:23pm UTC](https://discourse.julialang.org/t/unexpected-sensitivity-to-order-of-method-definitions/106510/9 "2023-11-21T14:23:07Z")

</div>

This example is in fact almost exactly demonstrated in [this comment](https://github.com/JuliaLang/julia/issues/9498#issuecomment-504363878) on [issue 9498](https://github.com/JuliaLang/julia/issues/9498), only the keyword argument has a default value. (The issue’s first example differs because the positional annotations can be different, so the methods shouldn’t override each other. `methods(f)` does show them as separate methods, and it’s more intuitive that a keyworded call can’t use a method with no keywords even if the positional annotation is more specific).

> **Redoing the demo with methods(f), keyword method first**
>
> ```julia
> julia> function f(a; b) a + b end;
> 
> julia> methods(f)
> # 1 method for generic function "f":
> [1] f(a; b) in Main at REPL[1]:1
> 
> julia> function f(a) a + a end;
> 
> julia> methods(f)
> # 1 method for generic function "f":
> [1] f(a; b) in Main at REPL[3]:1
> 
> julia> f(2), f(2; b=1)
> (4, 3)
> 
> ```

> **Demo but keyword method second in a different session**
>
> ```julia
> julia> function f(a) a + a end;
> 
> julia> methods(f)
> # 1 method for generic function "f":
> [1] f(a) in Main at REPL[1]:1
> 
> julia> function f(a; b) a + b end;
> 
> julia> methods(f)
> # 1 method for generic function "f":
> [1] f(a; b) in Main at REPL[3]:1
> 
> julia> f(2; b=1)
> 3
> 
> julia> f(2)
> ERROR: UndefKeywordError: keyword argument b not assigned
> 
> ```

If you’ve noticed, it certainly _seems_ as if the keyworded method actually had 2 methods, 1 throwing the error with no keywords. This would make a little more sense; in the first case, the error-throwing method was overwritten with a proper implementation, and vice versa in the second case. But where is it?

Let’s try just the keyworded method with a default value this time so we don’t throw errors. I will use `...` to omit some printed lines for brevity.

```julia
julia> function f(a; b=0) a+b end;

julia> methods(f)
# 1 method for generic function "f":
[1] f(a; b) in Main at REPL[14]:1

julia> @code_warntype f(1)
MethodInstance for f(::Int64)
  from f(a; b) in Main at REPL[14]:1
...
1 ─ %1 = Main.:(var"#f#3")(0, #self#, a)::Int64
└── return %1

julia> @code_warntype f(1; b=10)
MethodInstance for (::var"#f##kw")(::NamedTuple{(:b,), Tuple{Int64}}, ::typeof(f), ::Int64)
  from (::var"#f##kw")(::Any, ::typeof(f), a) in Main at REPL[14]:1
...
5 ┄ %16 = Main.:(var"#f#3")(b, @_3, a)::Int64
└── return %16

```

The keywordless call directly lists the 1 `f(a; b)` method, but the keyworded call _does not_. It lists a `var"#f##kw"` functor type. Both calls have the callee `var"#f#3`, only the keywordless call has the default value.

```julia
julia> methods(var"#f#3")
# 1 method for generic function "#f#3":
[1] var"#f#3"(b, ::typeof(f), a) in Main at REPL[14]:1

julia> var"#f#3"(10, f, 1)
11

julia> methods(var"#f##kw") # huh, constructor was omitted
# 0 methods for type constructor

julia> Base.issingletontype(var"#f##kw") # singleton.instance loophole!
true

julia> methods(var"#f##kw".instance)
# 1 method for anonymous function "f##kw":
[1] (::var"#f##kw")(::Any, ::typeof(f), a) in Main at REPL[14]:1

julia> var"#f##kw".instance((b=10,), f, 1)
11

```

So there you have it, keyworded calls are handled by a hidden singleton functor instance, and the keywordless calls are directly handled by the declared keyword method. The latter clashes with position-only methods, independently of the former. Both forward to a [callee, which is what you want to `@code_warntype`](https://github.com/JuliaLang/julia/issues/35717) to actually probe type instabilities (or use Cthulhu.jl to descend through callees).

---

<div class="post-metadata">

**Author:** ![tverho](https://avatars.discourse-cdn.com/v4/letter/t/839c29/32.png) [@tverho](https://discourse.julialang.org/u/tverho)\
**Post date:** [November 21, 2023, 8:27pm UTC](https://discourse.julialang.org/t/unexpected-sensitivity-to-order-of-method-definitions/106510/10 "2023-11-21T20:27:16Z")

</div>

Thanks for all the comments! Seems like this is a known issue and the problem is that handling keyword arguments is a little bit messy in Julia right now, making it harder to fix it. I’ll try not to use keyword arguments to affect dispatch (even though sometimes it would be quite convenient if it worked).

---

<div class="post-metadata">

**Author:** ![nsajko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nsajko/32/221187_2.png) [@nsajko](https://discourse.julialang.org/u/nsajko)\
**Post date:** [November 21, 2023, 8:31pm UTC](https://discourse.julialang.org/t/unexpected-sensitivity-to-order-of-method-definitions/106510/11 "2023-11-21T20:31:19Z")

</div>

> [@tverho](#):
>
> handling keyword arguments is a little bit messy in Julia right now

It’s fine, just don’t overwrite any methods. Overwriting methods is basically only useful while hacking anyway.

---

<div class="post-metadata">

**Author:** ![Benny](https://avatars.discourse-cdn.com/v4/letter/b/49beb7/32.png) [@Benny](https://discourse.julialang.org/u/Benny)\
**Post date:** [November 22, 2023, 8:03am UTC](https://discourse.julialang.org/t/unexpected-sensitivity-to-order-of-method-definitions/106510/12 "2023-11-22T08:03:03Z")

</div>

> [@tverho](#):
>
> I’ll try not to use keyword arguments to affect dispatch (even though sometimes it would be quite convenient if it worked).

It can be intimidating, but being aware of the limitations of mixing keywords and multimethod features helps a lot. One more thing I can mention is stick to the best practice of keeping keywords consistent across methods; ambiguities happen for the ones with the same number of positional arguments and related types, see the example below.

```julia
julia> bar(::Float64) = 0
bar (generic function with 1 method)

julia> bar(::Int; a) = 1 # if ::String, error below would not happen
bar (generic function with 2 methods)

julia> bar(::Number; a, b) = 2 # Int<:Number causes error below
bar (generic function with 3 methods)

julia> bar(1.0; a=0, b=0) # cannot use `bar(::Float64)`
2

julia> bar(1; a=0, b=0)
ERROR: MethodError: no method matching bar(::Int64; a=0, b=0)
Closest candidates are:
  bar(::Int64; a) at REPL[2]:1 got unsupported keyword argument "b"
  bar(::Number; a, b) at REPL[3]:1
  bar(::Float64) at REPL[1]:1 got unsupported keyword arguments "a", "b"

```

> [@tverho](#):
>
> handling keyword arguments is a little bit messy in Julia right now

Not sure if you also meant that you hope that keyword arguments can eventually distinguish methods by arity, type, and keyword, like how positional ones do by arity, type, and order, but just in case, that really is infeasible. At first it seems possible that keywords can be sorted by name, then it reduces to typical order-based dispatch. However, more methods that vary by keywords makes the above dispatch headaches even worse, and default values would cause combinatorial explosion. Right now, default values in positional arguments make multiple methods that scale linearly `N+1` e.g. `f(a=1, b=2)` makes `f(), f(a), f(a, b)`. If you apply this to keyword arguments that can be reordered, that becomes [`sum(factorial(N)/factorial(N-i) for i in 0:N)`](https://math.stackexchange.com/questions/813604/number-of-subset-permutations-for-a-set#comment1681353_813604) e.g. `f(;a=1, b=2)` makes `f(), f(;a), f(;b), f(;a, b), f(;b, a)`. The only way to avoid that combinatorial explosion is to force 1 method to handle all those possibilities and disallow other methods from handling any subset of them; silver lining is that the compiler can still specialize that 1 method on the various combinations of keywords you provide calls. It’s not a coincidence that languages with more flexible keyword arguments also lack multimethods or overloaded functions.