Why does defining a vararg method define a zero argument method?

Internals & Design
1

When defining a vararg method, a zero argument method is implicitly created:

julia> foo(args...) = sum([args...])
foo (generic function with 1 method)

julia> foo()
ERROR: MethodError: no method matching zero(::Type{Any})
Closest candidates are:
  zero(::Type{Union{Missing, T}}) where T at missing.jl:104
  zero(::Type{Dates.Time}) at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.5/Dates/src/types.jl:406
  zero(::Type{Dates.DateTime}) at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.5/Dates/src/types.jl:404
  ...
Stacktrace:
 [1] zero(::Type{Any}) at ./missing.jl:105
 [2] reduce_empty(::typeof(+), ::Type{Any}) at ./reduce.jl:311
 [3] reduce_empty(::typeof(Base.add_sum), ::Type{Any}) at ./reduce.jl:320
 [4] mapreduce_empty(::typeof(identity), ::Function, ::Type{T} where T) at ./reduce.jl:343
 [5] reduce_empty(::Base.MappingRF{typeof(identity),typeof(Base.add_sum)}, ::Type{Any}) at ./reduce.jl:329
 [6] reduce_empty_iter at ./reduce.jl:355 [inlined]
 [7] mapreduce_empty_iter(::Function, ::Function, ::Array{Any,1}, ::Base.HasEltype) at ./reduce.jl:351
 [8] _mapreduce(::typeof(identity), ::typeof(Base.add_sum), ::IndexLinear, ::Array{Any,1}) at ./reduce.jl:400
 [9] _mapreduce_dim at ./reducedim.jl:318 [inlined]
 [10] #mapreduce#620 at ./reducedim.jl:310 [inlined]
 [11] mapreduce at ./reducedim.jl:310 [inlined]
 [12] _sum at ./reducedim.jl:727 [inlined]
 [13] _sum at ./reducedim.jl:726 [inlined]
 [14] #sum#627 at ./reducedim.jl:722 [inlined]
 [15] sum at ./reducedim.jl:722 [inlined]
 [16] foo() at ./REPL[1]:1
 [17] top-level scope at REPL[2]:1

In my experience, this is usually not what you want. In order to avoid this, you have to do something like this:

julia> foo(x, y...) = sum([x, y...])
foo (generic function with 1 method)

julia> foo()
ERROR: MethodError: no method matching foo()
Closest candidates are:
  foo(::Any, ::Any...) at REPL[1]:1

Another downside to implicitly defining the zero argument method is that it is easy to accidentally engage in type piracy when you are extending a method:

struct A
    x::Int
end

Base.max(as::A...) = max(map(a -> a.x, as)...)

This defines a Base.max() method, which is type piracy. Not to mention the unintended zero argument method causes a stack overflow:

julia> max(A(2), A(3))
3

julia> max(A(2))
2

julia> max()
ERROR: StackOverflowError

I’ve listed here a few costs to implicitly defining a zero argument method when defining a vararg function. Are there any benefits?

3 Likes
2

I disagree, it is absolutely what I normally want. And if not, how would I express “zero or more inputs”? And what about

f(x, y; kwargs...)

Should it require at least one key-value pair?

6 Likes
3

This one is simple to deal with, you just define a new method that takes zero inputs (and let the vararg take care of one or more). The remaining criticism is very on point, however.

4

That doesn’t seem so easy to me, very awkward at least.

There’s also the asymmetry between slurp and splat: you could splat an empty container, but not slurp into an empty one? Or should one also disable splatting for length-1 containers?

To me, slurped arguments are for optional inputs, not “optional, but at least one”. It would mess up a lot of code (even disregarding the massive breakage).

3 Likes
5

I agree that vararg keyword arguments need to accept zero or more arguments. But I’d like to see an example where implicitly defining the zero-positional-arguments method is actually useful.

Here’s another cumbersome case that I stole from @pfitzseb:

julia> f(xs::Int...) = 1
f (generic function with 1 method)

julia> f(xs::Float64...) = 2
f (generic function with 2 methods)

julia> f()
ERROR: MethodError: f() is ambiguous. Candidates:
  f(xs::Int64...) in Main at REPL[1]:1
  f(xs::Float64...) in Main at REPL[2]:1
Possible fix, define
  f()
2 Likes
6

To me, optional arguments are expressed like this:

foo(x, y=1) = x + y
7

That’s not what I normally use slurp for, most of the time it’s something like this

foo(x, y, args...) = bar(2x+y^2, args...)

I think this would become really inconvenient with disabled zero-length slurping.

It would also just seem like an arbitrary and inconsistent limitation.

5 Likes
8

Ah, I suppose I’ve mainly been thinking about the foo(args...) case. If you’re using foo(x, y, args...) it’s perfectly reasonable to accept zero positional arguments. However, in the foo(args...) case, the implicit definition of a zero-positional-arguments foo() method causes all kinds of subtle problems. Perhaps a warning should be added to the manual.

1 Like
9

This seems a clear case in which the function f should either take a Vector/“require one positional argument” or f() is never intended to be called (if this annoys you, you can require the first positional argument manually). Function with methods specialized for different types either specialize on a required field, or have no reason to ever be called without arguments.

1 Like
10

This example would also be a method error if ... required one or more arguments.

3 Likes
11

Isn’t the canonical solutions to this to just do:

foo(arg1, args...) = something with arg1 and args

And it’s explicit that you mean one or more args.

2 Likes
12

Yeah, I mentioned that pattern in my original post.

1 Like
13

Oh right, with foo(x, y...)

I don’t see how this could be any other way with consistent slurp behavior across the language. It has to always mean “maybe some arguments” or “one argument or more” in all cases.

14

This reminds me of Mathematica, which has separate constructs for one-or-more vs. zero-or-more arguments.

Here’s one-or-more, using 2 underscores (lifted directly from their documentation):

In[1]:= f[x__] := Length[{x}]
In[2]:= {f[x, y, z], f[]}
Out[2]= {3, f[]}

Here’s zero-or-more, with three underscores:

In[1]:= f[x___] := p[x, x]
In[2]:= {f[], f[1], f[1, a]}
Out[2]= {p[], p[1, 1], p[1, a, 1, a]}

There are definite use cases for either, although the underscores can be tricky to count. In Julia, one could imagine a .... vs. ... or somesuch. There would be risk of confusing them, but at least dots are easier to count than underscores.

4 Likes
15

If args... slurp syntax in a method signature only matched one or more arguments, would you only be allowed to splat collections with at least one item? It seems like it would be awkward and confusing if those were asymmetrical.

3 Likes
16

While I can understand the frustration with the original max example, I think there are other issues with it that zero argument slurping is being scapegoated for. Some are problems with the definition itself while others are issues with the language. One problem is that this example is a pseudo-recursive definition that calls a different method of the same function, which should generally be done only with extra care and this definition gets it wrong. A closely related issue is that the core method of max is binary — max(a, b) — and the varargs methods are generically defined in terms of that. If you want to overload the behavior of max for a custom type, the right way to do it is to specialize the “core” method — ie the two argument one — and let that behavior flow out to the generic varargs methods naturally from there. So what really trips this definition up is that it tries to specialize by overwriting the more general definition directly, which leads to ambiguity problems. If method ambiguity was defined strictly in terms of subtyping of signatures, then this definition would lead to a method error since it is neither strictly more specific nor strictly less specific than the existing generic varargs max method in the zero argument case. So the language shortcomings that this highlights are:

  1. Lack of formal interfaces and restrictions on how one can extend generic functions like max. If you were only allowed to extend it by specializing the two-argument method then you would have been lead to the right solution here. The cost would be that there may be situations where one need to directly specialize the more general method and if we disallowed that it could be problematic (limit what people can do or require a way to let someone say “no, I really want to do this”).

  2. Heuristics on method specificity are imperfect and cannot get every case right. Subtype-based specificity is strictly correct but using that gets pretty annoying for varargs methods. In this case the heuristic is that if there are overlapping varargs methods where the element type of one is more restricted than the other, then the more restricted method is considered more specific. There’s a case to be made that the zero-argument case in such situations should be considered ambiguous.

So while there are issues highlighted by this example, I don’t necessarily think the right conclusion is that slurping should not match zero arguments.

7 Likes
17

Sorry, I didn’t mean to open up a whole can of worms with that max example. That was just a minimal example that I hacked together in the REPL. Clearly that example has some issues.

This issue was actually brought to my attention when I was looking at some code from MLJBase.jl. The VS Code linter was complaining about this:

image
image1168Ă—556 73 KB

The reason the linter was complaining is because it turns out that

Base.cat(us::UnivariateFiniteArray{S,V,R,P,N}...;
         dims::Integer) where {S,V,R,P,N}

is actually engaging in type piracy, since it is implicitly defining the zero argument Base.cat() method.

To clarify, I’m not really arguing that slurping shouldn’t ever be able to slurp zero arguments. If your method definition is foo(x, y...), then it’s reasonable to slurp zero arguments. But if your method definition is foo(x...), then it’s less reasonable to slurp zero arguments and it can be problematic to implicitly define the zero argument foo() method.

However, I admit that special casing foo(x...) to not allow slurping zero arguments is a little awkward and inconsistent.

2 Likes
18

If we can agree on syntax to specify “one or more” arguments, I don’t think implementing it would be too tough. I’ve always found the explicit workaround pretty ugly.

5 Likes
19

Probably I am missing something here, but I thought

f(x, xs...) = ...

was precisely for that.

1 Like
20

I think the foo(x, xs...) syntax is the less-than-beautiful workaround that he was referring to. A devoted syntax specifically for one-or-more slurping could be something like this:

foo(x~~~) = ...

I’m not saying that’s a good one, just an example.

However, I’m ok with the status quo. Maybe I’ll make a PR to the manual that adds some explanation about the zero argument case.

1 Like