Mystery of kwargs

General Usage
1

I am completely flummoxed by using a dict as the kwargs input in a varargs function that uses keyword arguments. Read the manual many times but it leaves out a critical thing: how does the callee unpack the varargs? The manual says I should be able to pass in a dict and in the callee, it should appear as an iterator over a named tuple.

function m(; kwargs...)
     println(kwargs)
     println(kwargs.a)
     println(a)
end

Construct a dict to pass to m:

foo = Dict(:a=>1, :b=>2, :c=>3)

Let’s call m passing in foo. Somehow, in the body of m we must be able to refer to something that was passed in by some variable name. This is opaque to me. We can’t refer to kwargs or do anything with it. We can’t access its member names. What can we do with kwargs or its members within m? I can’t even use the function. Am I supposed to splat the input argument when I call–that doesn’t work either?

julia> m(foo)
ERROR: MethodError: no method matching m(::Dict{Symbol,Int64})
Closest candidates are:
  m(; kwargs...) at REPL[88]:2

Let’s splat the input argument:

julia> m(foo...)
ERROR: MethodError: no method matching m(::Pair{Symbol,Int64}, ::Pair{Symbol,Int64}, ::Pair{Symbol,Int64})

I can’t even get passed the method error. I am missing something very basic here.

1 Like
2

No you need to define the actual keyword arguments to the method, unless you are just passing them along to an inner function.

function m(;a=0,b=0,c=0)
    println(a)
    println((a,b,c))
end

and secondly, in a situation like this you need to place a semicolon before the keywords

m(;foo...)
6 Likes
3

Aside from the answer above, another pretty idiomatic way to do this is with the @unpack macro from Parameters.jl:

function m(;kwargs...)
    @unpack a,b,c = kwargs
    println(a)
    println((a,b,c))
end
6 Likes
4

OP in addition the previous answer I think you want

julia> function m(; kwargs...)
           (;kwargs...)
       end

for turning arbitrary keyword arguments into a named tuple.

2 Likes
5

I guess in the interest of completeness I should say that you can actually do it the way you tried, but you accessed them incorrectly

function m2(; kwargs...)
            println(kwargs)
            println(kwargs[:a])
       end
6

This make sense. So simple. It’s got nothing to do with varargs and varargs functions.

It’s just that we call a function that needs keyword arguments with a collection that uses names (dict or named tuple) that match the input arguments and splat it. It’s the splat the matches up the names in the pairs to the input arguments because the function “sees” the pairs, not the unsplat’ed collection.

So, why not document it that way? Seems like the doc sort of munges varargs and keyword args. Or I munged them in my brain. (OK–the latter.) I’ll try a pull req on the doc with a couple of small clarifying examples.

And a varargs function is a totally different beast. It has a variable number of inputs (with names or not) and since the callee (the function definition, of course) doesn’t know in advance how many args to expect, it can’t give them names–it must iterate until the passed in iterator is exhausted.

Thanks–I’ll do it this simple, obvious way. Of course, it worked because it was how I started before I confused myself with a look at the manual.

1 Like
7

kwargs are stored in a datastructure called Iterators.Pairs t make a couple of interfaces more convenient. You can explore that like this:

julia> foo(;kwargs...) = dump(kwargs)
foo (generic function with 1 method)

julia> foo(a=1, b=2, c=3)
Base.Iterators.Pairs{Symbol,Int64,Tuple{Symbol,Symbol,Symbol},NamedTuple{(:a, :b, :c),Tuple{Int64,Int64,Int64}}}
  data: NamedTuple{(:a, :b, :c),Tuple{Int64,Int64,Int64}}
    a: Int64 1
    b: Int64 2
    c: Int64 3
  itr: Tuple{Symbol,Symbol,Symbol}
    1: Symbol a
    2: Symbol b
    3: Symbol c

If we stare at this for a bit, we see that we can get out the original NamedTuple by calling kwargs.data.

julia> bar(;kwargs...) = kwargs.data
bar (generic function with 1 method)

julia> bar(a=1, b=2, c=3)
(a = 1, b = 2, c = 3)

julia> bar(a=1, b=2, c=3).a
1
3 Likes
8

This is also really, really helpful. Even with a varargs function, we can recover the input and it’s field names–so we don’t need a function signature with the keyword arguments (though we’d still need to know what the inbound names are). That’s awesome. Using the keywork args in the function signature is still helpful as documentation if we don’t actually need a variable number of arguments.

I was wrong again–there is a bridge here between kwargs with keys and varargs.

Where’s the right place to figure this out? Source?

Frustration and helpful folks lead to learning.

9

I learned most of what I know though a combination of just exploring at the REPL (making heavy use of the dump function and function docstrings) and asking around for help.

I generally find ‘reading the source’ to be somewhat unhelpful unless the source code is very naive and basic.

For instance, what I posted above is essentially the exact process of how I learned about the kwargs storage. Someone asked a similar question some moons ago and I was playing around with it, made a function that slurps up kwargs and returns them, then called dump on the output and saw that the original NamedTuple was hidden in the Pairs structure.

2 Likes
10

Definitely recommend the dump function! Very useful.

1 Like
11

Sounds like a good outcome to me! I noticed in the keyword arguments section of the manual they don’t have any examples with f(; kwargs...) pattern, so that could be a good addition. They do with have f(x; y=0, kwargs...), but I think it’s not always obvious you need the ; even when there aren’t any non-keyword-argument arguments to the function.

4 Likes
12

Here is my code to understand the concept of kwargs...:

# --- ----------------------------------------------------------------------------------------------------------------------
# Example of function with "kwargs..." argument
# kwargs are immutable key-value iterator over a named tuple or dictionaries with keys of Symbol. 
# There exist two options to hand over a series of named parameters: "kwargs..."
# a) named tuple, b) dictionaries with symbols as keys
# They can be passed as keyword arguments using a semicolon in a call, e.g. f(x, z=1; kwargs...)
# 
# --------------------------------------------------------------------------------------------------------------------------
function _foo(; kwargs...)
    # (;kwargs...)
    if haskey(kwargs, :m)
        # println("kwargs[:m]: ")
        println("kwargs[:m]: ", kwargs[:m])
        _m = kwargs[:m]
    else
        println("Key :m not included!")
        _m = 2.0
    end
    if haskey(kwargs, :n)
        # println("kwargs[:m]: ")
        println("kwargs[:m]: ", kwargs[:n])
        _n = kwargs[:n]
    else
        println("Key :m not included!")
        _n = 2.0
    end
    if haskey(kwargs, :o)
        # println("kwargs[:m]: ")
        println("kwargs[:o]: ", kwargs[:o])
        _o = kwargs[:o]
    else
        println("Key :o not included!")
        _o = "o"
    end
    _kwargs_out = (m = _m, n = _n, o = _o)
    return _kwargs_out
end

foo_kwargs_NT = (m = 1.2, n = 3.2)
foo_kwargsDict = Dict(:m => 1.2, :n => 3.2)

foo_kwargs_NT[:m]
foo_kwargsDict[:m]

haskey(foo_kwargs_NT, :m)
haskey(foo_kwargsDict, :m)

println("kwargs = named tuple:       ", '-'^75)
out_1 = _foo(; foo_kwargs_NT...);
println("kwargs = Dict with symbols: ", '-'^75)
out_2 = _foo(; foo_kwargsDict...);
println("kwargs omitted:             ", '-'^75)
out_3 = _foo()
println("kwargs as one named parameter: ", '-'^73)
out_4 = _foo(; o = "O la la!")
println('-'^103)
println("out_1: ", out_1)
println("out_2: ", out_2)
println("out_3: ", out_3)
println("out_4: ", out_4)
println('-'^103)