# Weird macro behavior trying to simulate keyword arguments

**URL:** <https://discourse.julialang.org/t/weird-macro-behavior-trying-to-simulate-keyword-arguments/17800>\
**Category:** General Usage\
**Created:** [November 21, 2018, 4:43am UTC](https://discourse.julialang.org/t/weird-macro-behavior-trying-to-simulate-keyword-arguments/17800 "2018-11-21T04:43:14Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [November 21, 2018, 4:43am UTC](https://discourse.julialang.org/t/weird-macro-behavior-trying-to-simulate-keyword-arguments/17800/1 "2018-11-21T04:43:14Z")

</div>

DataFramesMeta’s macros are weird, because they want to behave like functions. `@transform(df, y = 5)`, but they are still macros.

If `@transform` were a function, it would be implemented like `transform(df; kwargs...)`. Indeed, this is how the functions that `@transform` calls work.

So I thought, in [DataFramesMeta #117](https://github.com/JuliaData/DataFramesMeta.jl/issues/117). So I thought that it would be nice to have `pairs` of keyword arguments work in `@transform` as well so you could get run-time keyword argument evaluation `@transform(df; x => mean(:x1)` as well.

It turns out this isn’t so easy, and definitions like `macro foo(args...; kwars...)` are disallowed. This is understandable, since macros work with `Expr` and therefor keyword arguments dont really make sense.

No big deal, one can just inspect the AST inputted into a macro call and make `pair`s of keyword arguments just work, right? This is where unexpected behavior comes in.

```julia
macro foo(args...)
    dump(ars)
end

```

Returns

```julia
julia> @foo(a = 1; b = 2)
Tuple{Expr,Expr}
  1: Expr
    head: Symbol parameters
    args: Array{Any}((1,))
      1: Expr
        head: Symbol kw
        args: Array{Any}((2,))
          1: Symbol b
          2: Int64 2
  2: Expr
    head: Symbol =
    args: Array{Any}((2,))
      1: Symbol a
      2: Int64 1

```

However

```julia
macro bar(x, args...)
    dump(args)
end

```

returns

```julia
julia> @bar(x, a = 1; b = 2)
Tuple{Symbol,Expr}
  1: Symbol x
  2: Expr
    head: Symbol =
    args: Array{Any}((2,))
      1: Symbol a
      2: Int64 1

```

When you have a macro where everything is splatted, the collected expression includes everything between the parentheses. But when you have a multi-argument macro, where the second argument is splatted, the collected expression only goes until the first semi-colon.

This seems like unexpected and inconsistent behavior. Should I file an issue?

Seems to be some discussion of similar behavior [here](https://github.com/MikeInnes/MacroTools.jl/issues/4) in MacroTools.jl.

---

<div class="post-metadata">

**Author:** ![mauro3](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mauro3/32/292_2.png) [@mauro3](https://discourse.julialang.org/u/mauro3)\
**Post date:** [November 21, 2018, 7:38am UTC](https://discourse.julialang.org/t/weird-macro-behavior-trying-to-simulate-keyword-arguments/17800/2 "2018-11-21T07:38:17Z")

</div>

All the kwargs get collected in the first argument `x`, all the rest gets collected in `args`. This is indeed a bit weird. Maybe worth opening an issue. But this couldn’t be changed until Julia 2.0.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [November 21, 2018, 9:24am UTC](https://discourse.julialang.org/t/weird-macro-behavior-trying-to-simulate-keyword-arguments/17800/3 "2018-11-21T09:24:05Z")

</div>

> [@mauro3](#):
>
> But this couldn’t be changed until Julia 2.0.

Not unless it is considered a bug, and the changes [don’t break any registered package](https://discourse.julialang.org/t/proposed-release-process-and-schedule/15623).

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [November 21, 2018, 12:51pm UTC](https://discourse.julialang.org/t/weird-macro-behavior-trying-to-simulate-keyword-arguments/17800/4 "2018-11-21T12:51:01Z")

</div>

> [@pdeffebach](#):
>
> When you have a macro where everything is splatted, the collected expression includes everything between the parentheses. But when you have a multi-argument macro, where the second argument is splatted, the collected expression only goes until the first semi-colon.

What’s the inconsistency here? The keyword argument is always passed in first, which is strange I agree but not inconsistent, so when you splat everything, you get the keyword argument followed by the positional ones and if you supply one normal argument first, that argument gets the, well, first argument, namely the keyword, and the splatting gets the rest, namely the positional ones. Essentially, if you define `bar` as `dump((x, args...))` it should behave exactly the same as `foo`. This seems to be the case so the two are consistent.

> [@Tamas\_Papp](#):
>
> changes [don’t break any registered package](https://discourse.julialang.org/t/proposed-release-process-and-schedule/15623)

As I said, the only strange thing here is the argument where keyword arguments are passed in. Changing this is highly breaking.

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [November 21, 2018, 2:20pm UTC](https://discourse.julialang.org/t/weird-macro-behavior-trying-to-simulate-keyword-arguments/17800/5 "2018-11-21T14:20:34Z")

</div>

Thanks for the answer! `dump((x, args...))` gets the first argument as the `:parameters`-annotated keyword arguments.

I will admit that this is a bit of a pain. The relevant code to work with is here:

```julia
function transform_helper(x, args...)
    quote
        $transform($x, $(map(args) do kw
            Expr(:kw, kw.args[1], with_anonymous(kw.args[2]))
        end...) )
    end
end

```

And adding some keyword-argument-type thing would break this as now `x` becomes the `:parameters` expression, which is counter-intuitive.

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [November 21, 2018, 3:09pm UTC](https://discourse.julialang.org/t/weird-macro-behavior-trying-to-simulate-keyword-arguments/17800/6 "2018-11-21T15:09:03Z")

</div>

That I agree. It even had [real consequences](https://github.com/JuliaLang/julia/issues/9535) that I complained about. Unfortunately, changing this will be very breaking and I’m not sure if there can be a way to give a useful depwarn for macros…

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [November 21, 2018, 3:32pm UTC](https://discourse.julialang.org/t/weird-macro-behavior-trying-to-simulate-keyword-arguments/17800/7 "2018-11-21T15:32:05Z")

</div>

Is there any way to leverage the way functions automatically parse this so I can avoid doing a bunch of `=>` to `=` transitions and sorting through the syntax tree?

Or is it all done in lisp?

---

<div class="post-metadata">

**Author:** ![tkoolen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkoolen/32/1603_2.png) [@tkoolen](https://discourse.julialang.org/u/tkoolen)\
**Post date:** [November 21, 2018, 3:42pm UTC](https://discourse.julialang.org/t/weird-macro-behavior-trying-to-simulate-keyword-arguments/17800/8 "2018-11-21T15:42:29Z")

</div>

Tangentially related, the way I’ve done keyword arguments with macros in the past is to have the macro create an expression that calls a proper keyword argument function which returns the ‘completed’ set of parameters. See

> <https://github.com/tkoolen/LoopThrottle.jl/blob/3d90af9db9c14fc274f1d110948cef21d6ffa0b3/src/LoopThrottle.jl#L64>

(The whole package is just 100 lines with 40 lines of docstring, so it should be pretty obvious what’s going on.)

---

<div class="post-metadata">

**Author:** ![pdeffebach](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pdeffebach/32/10320_2.png) [@pdeffebach](https://discourse.julialang.org/u/pdeffebach)\
**Post date:** [November 21, 2018, 6:37pm UTC](https://discourse.julialang.org/t/weird-macro-behavior-trying-to-simulate-keyword-arguments/17800/9 "2018-11-21T18:37:38Z")

</div>

This is interesting, thanks!

My use-case is probably something a bit more abstract, since we would just have `kwargs...` splatted and not know what they will be.

It seems like a `MacroKeywordArguments` could be a useful package.