# Metaprogramming: adding an arg to a function

**URL:** <https://discourse.julialang.org/t/metaprogramming-adding-an-arg-to-a-function/19262>\
**Category:** General Usage\
**Tags:** metaprogramming\
**Created:** [January 4, 2019, 8:43am UTC](https://discourse.julialang.org/t/metaprogramming-adding-an-arg-to-a-function/19262 "2019-01-04T08:43:21Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![Philippe\_Maincon1](https://avatars.discourse-cdn.com/v4/letter/p/ec9cab/32.png) [@Philippe\_Maincon1](https://discourse.julialang.org/u/Philippe_Maincon1)\
**Post date:** [January 4, 2019, 8:43am UTC](https://discourse.julialang.org/t/metaprogramming-adding-an-arg-to-a-function/19262/1 "2019-01-04T08:43:21Z")

</div>

Julia 1.0.

Me and metaprogramming, I don’t know, I am real slow at this!

I want to write a macro that, given a function definition

```julia
function f(x,y)
do something
end

```

will generate

```julia
function f(store,x,y)
do something
do something with store
end

```

now “x,y” is an array of expressions, I do not see how to put that back into the expression I am generating. Here is my MNWE:

```julia

macro store(ex) # given a function, create the same function with the additional first argument "store"
        local name = ex.args[1].args[1]            
        local arglist = ex.args[1].args[2:end] # NB: array of Expr  
        local body = ex.args[2]
        out= quote
            function $(esc(name))(store,$arglist) # almost! arg is an array, and that does not work!
                $(esc(body))
                # do something with "store" here, not relevant for MWE
            end
        end
        return out
end

@store function f(x)
    y = x
    for i = 1:3
        y = 3y
    end
    z = y+2
    return 2z
end

store()=nothing
f(store,3)

```

---

<div class="post-metadata">

**Author:** ![mauro3](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mauro3/32/292_2.png) [@mauro3](https://discourse.julialang.org/u/mauro3)\
**Post date:** [January 4, 2019, 9:04am UTC](https://discourse.julialang.org/t/metaprogramming-adding-an-arg-to-a-function/19262/2 "2019-01-04T09:04:23Z")

</div>

This seems to work:

```julia
macro store(ex) # given a function, create the same function with the additional first argument "store"
    name = esc(ex.args[1].args[1])
    arglist = esc.(ex.args[1].args[2:end]) # NB: array of Expr
    body = esc(ex.args[2])
    out= quote
        function $(name)(store, $(arglist...)) # almost! arg is an array, and that does not work!
            $(body)
            # do something with "store" here, not relevant for MWE
        end
    end
    return out
end

@store function f(x)
    y = x
    for i = 1:3
        y = 3y
    end
    z = y+2
    return 2z
end

f(nothing, 3)

```

Note the use of the `...` inside the `$`-interpolation.

---

<div class="post-metadata">

**Author:** ![Philippe\_Maincon1](https://avatars.discourse-cdn.com/v4/letter/p/ec9cab/32.png) [@Philippe\_Maincon1](https://discourse.julialang.org/u/Philippe_Maincon1)\
**Post date:** [January 4, 2019, 9:42am UTC](https://discourse.julialang.org/t/metaprogramming-adding-an-arg-to-a-function/19262/3 "2019-01-04T09:42:29Z")

</div>

Excellent!

It took me time to notice the esc.() call, which is also quite important.

Thank you, good teacher!

---

<div class="post-metadata">

**Author:** ![mauro3](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mauro3/32/292_2.png) [@mauro3](https://discourse.julialang.org/u/mauro3)\
**Post date:** [January 4, 2019, 10:52am UTC](https://discourse.julialang.org/t/metaprogramming-adding-an-arg-to-a-function/19262/4 "2019-01-04T10:52:04Z")

</div>

You’re welcome. (Also mark it as solved.)

---

<div class="post-metadata">

**Author:** ![Philippe\_Maincon1](https://avatars.discourse-cdn.com/v4/letter/p/ec9cab/32.png) [@Philippe\_Maincon1](https://discourse.julialang.org/u/Philippe_Maincon1)\
**Post date:** [January 5, 2019, 5:29am UTC](https://discourse.julialang.org/t/metaprogramming-adding-an-arg-to-a-function/19262/5 "2019-01-05T05:29:59Z")

</div>

OK, will keep “mark as solved” in mind. A continuation question. Now that things work, I package them in a module, and again, I get baffled, here is my MNWE.

```julia
__precompile__ ()
module InsiderMVE
    using Printf
    export spy, @store, store

    function spy(targets)
        vals=[]
        tags=[]
        function store(val,tag)
            if tag ∈ targets
                push!(vals,copy(val))
                push!(tags,tag)
            end
        end
        store(val)=store(val,:unspecified)
        function get()
            return (vals,tags)
        end
        return store,get
    end

    macro store(ex)
        if ex.head== :(=) # @store y = expr
            local var = ex.args[1] # :y
            return quote
                $(esc(ex)) # y = expr
                store($(esc(var)), $(Meta.quot(var))) # store(y,:y)
            end
        elseif ex.head== :function # @store function f(x::float64,y) return x*y end
            foo = esc( ex.args[1].args[1]) # :f
            arg = esc.(ex.args[1].args[2:end]) # [:(x::float64),:y]
            body = esc( ex.args[2])
            out= quote
                function $foo(store,$(arg...)) # f(store,,) x::float64,y) return x*y end
                    $body
                end
            end
            return out
        end
    end
end #module spying

```

which I test with (Julia 1.0.1)

```julia
using InsiderMVE

@store function f(x)
    y = x
    for i = 1:3
        @store y = 3y
    end
    @store z = y+2
    return 2z
end

(s,results)=spy([:y,:z])
r=f(s,3)
(vals,tags)=results()
display(vals)
display(tags)

```

causing error

```julia
ERROR: LoadError: UndefVarError: store not defined
Stacktrace:
 [1] macro expansion at C:\Users\philippem\home\GIT\julia\modules_1_0\src\Insider.jl:59 [inlined]
 [2] macro expansion at C:\Users\philippem\home\GIT\julia\modules_1_0\test\test_insiderMVE.jl:7 [inl
ined]
 [3] f(::Function, ::Int64) at C:\Users\philippem\home\GIT\julia\modules_1_0\src\Insider.jl:68
 [4] top-level scope at none:0
in expression starting at C:\Users\philippem\home\GIT\julia\modules_1_0\test\test_insiderMVE.jl:14

```

Now, when “store” is undefined - in what scope, at what time?

: )

---

<div class="post-metadata">

**Author:** ![ffevotte](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ffevotte/32/6587_2.png) [@ffevotte](https://discourse.julialang.org/u/ffevotte)\
**Post date:** [January 5, 2019, 3:09pm UTC](https://discourse.julialang.org/t/metaprogramming-adding-an-arg-to-a-function/19262/6 "2019-01-05T15:09:26Z")

</div>

This happens during your call to `f(s, 3)`, so that we know that macro expansion went well (at least did not error out). What happens here is that `store` seems to be undefined within the body of `f`.

As is often the case with macros, a good debugging method consists in checking that the macro expansion does what you want it to do.

In your case:

```julia
julia> @macroexpand @store function f(x)
           y = x
           for i = 1:3
               @store y = 3y
           end
           @store z = y+2
           return 2z
       end

```

```julia
# (output manually shortened by removing lines indications)
quote
    function f(#7#store, x)
        begin
            y = x
            for i = 1:3
                begin
                    y = 3y
                    (Main.store)(y, :y)
                end
            end
            begin
                z = y + 2
                (Main.store)(z, :z)
            end
            return 2z
        end
    end
end

```

What we see here is that the first argument of `f` is a specific symbol `#7#store` designed to avoid clashing with the rest of the code (as part of macro hygiene). Whereas calls to `store(y, :y)` and `store(z, :z)` refer to a (non-existing) `Main.store` symbol.

I’m trying to think of a good way to ensure that the same symbol is used at both places, but I’m currently not sure how to do this wile keeping your current approach where there are 3 calls to `@store`, separately instrumenting the function definition and some instructions in it…

---

<div class="post-metadata">

**Author:** ![ffevotte](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ffevotte/32/6587_2.png) [@ffevotte](https://discourse.julialang.org/u/ffevotte)\
**Post date:** [January 5, 2019, 3:22pm UTC](https://discourse.julialang.org/t/metaprogramming-adding-an-arg-to-a-function/19262/7 "2019-01-05T15:22:12Z")

</div>

To illustrate the problem I mention above, consider the following macro:

```julia
macro store(ex)
    store = :store_123456 #<-- WARNING: this is not hygienic!
    if ex.head== :(=) # @store y = expr
        local var = ex.args[1] # :y
        return quote
            $(esc(ex)) # y = expr
            $(esc(store))($(esc(var)), $(Meta.quot(var))) # store(y,:y)
        end
    elseif ex.head== :function # @store function f(x::float64,y) return x*y end
        foo = esc( ex.args[1].args[1]) # :f
        arg = esc.(ex.args[1].args[2:end]) # [:(x::float64),:y]
        body = esc( ex.args[2])
        out= quote
            function $foo($(esc(store)),$(arg...)) # f(store,,) x::float64,y) return x*y end
                $body
            end
        end
        return out
    end
end

```

It ensures that the expansions of

```julia
@store y = 3y

```

and

```julia
@store function f()
   # ...
end

```

refer to the same symbol (`store_123456`). But this macro is not hygienic: if the code you write in the definition of `f` happens to refer to a symbol `store_123456`, then you have a collision between the code introduced by your macro and the original definition of `f`.

`gensym` is a standard way of generating symbols which will not conflict with other variables. But if you have 3 separate macro expansions, then you’ll get 3 separate symbols and you’ll be back to the current problem (although in a more hygienic way)…

---

<div class="post-metadata">

**Author:** ![ffevotte](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ffevotte/32/6587_2.png) [@ffevotte](https://discourse.julialang.org/u/ffevotte)\
**Post date:** [January 5, 2019, 8:45pm UTC](https://discourse.julialang.org/t/metaprogramming-adding-an-arg-to-a-function/19262/8 "2019-01-05T20:45:54Z")

</div>

This macro should do what you want, and still be hygienic.

```julia
macro store(ex)
    # Auxilliary function to recursively transform the body

    # Most AST nodes should probably left untouched, but maybe not all. Let's
    # explicitly list what we don't want to transform...
    aux(ex::Union{Number,LineNumberNode}) = ex
    # ... and error out if we've forgotten anything.
    aux(ex) = error("Oops, I forgot something: ", ex)

    # Symbols coming from "user" code should be escaped
    aux(s::Symbol) = esc(s)

    # Expressions are transformed
    aux(ex::Expr) = if ex.head == :macrocall && ex.args[1] == Symbol("@store")
        # If this is a macro call of the type:
        #
        # @store VAR = EXPR
        #
        # we want to transform it to something like:
        #
        # begin
        # VAR = EXPR
        # store(VAR, :VAR)
        # end
        #
        ex.args[2] :: LineNumberNode
        assignment = ex.args[3]
        @assert assignment.head == :(=)
        var = assignment.args[1]
        quote
            $(aux(assignment))
            store($(esc(var)), $(Meta.quot(var)))
        end
    else
        # In the general case, expressions are just recursively transformed
        Expr(ex.head, aux.(ex.args)...)
    end

    # @store should be called on a function definition
    @assert ex.head == :function
    foo = esc(ex.args[1].args[1])
    arg = esc.(ex.args[1].args[2:end])
    body = ex.args[2]
    quote
        function $foo(store, $(arg...))
            $(aux(body))
        end
    end
end

```

In particular, for your test case, it expands in the following way, where you see that the introduced `store` symbol (named `#311#store` in this particular case) can not conflict with “user-written” code.

```julia
julia> @macroexpand @store function f(x)
           y = x
           for i = 1:3
               @store y = 3y
           end
           @store z = y+2
           return 2z
       end

```

```julia
# Output manually edited to remove LineNumberNode annotations
quote
    function f(#311#store, x)
        begin
            y = x
            for i = 1:3
                begin
                    y = 3y
                    #311#store(y, :y)
                end
            end
            begin
                z = y + 2
                #311#store(z, :z)
            end
            return 2z
        end
    end
end

```

In order to keep your original syntax, the inner `@store` macro calls are used as annotations during the expansion of the outer `@store` call. They are removed from the expansion, so that they don’t have a chance to get expanded themselves. To put it another way, although there seem to be 3 distinct macro calls in this example, only the outer expansion takes place: the two inner ones are removed before they can be expanded. I’m not sure whether this is a very Julian way of doing things…

  

In any case, using this macro, your initial example should now work. BTW, I don’t think this specific problem had anything to do with modules…

---

<div class="post-metadata">

**Author:** ![Philippe\_Maincon1](https://avatars.discourse-cdn.com/v4/letter/p/ec9cab/32.png) [@Philippe\_Maincon1](https://discourse.julialang.org/u/Philippe_Maincon1)\
**Post date:** [January 7, 2019, 8:08am UTC](https://discourse.julialang.org/t/metaprogramming-adding-an-arg-to-a-function/19262/9 "2019-01-07T08:08:27Z")

</div>

Aha, thank you for the insight: My fundamental problem is that I do code transformation through multiple macro calls, creating the problem with non-matching symbols.

I still have to read your last post, but my comment at this point in time is that my `@store y = 3y` should not generate a call to a function, but just leave a tag (somehow!), so that `@store function...` should do all the code generation (change to function header and calls to `@store`… in one go and with one consistent `f`-local name for function `store`.

---

<div class="post-metadata">

**Author:** ![ffevotte](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ffevotte/32/6587_2.png) [@ffevotte](https://discourse.julialang.org/u/ffevotte)\
**Post date:** [January 7, 2019, 8:27am UTC](https://discourse.julialang.org/t/metaprogramming-adding-an-arg-to-a-function/19262/10 "2019-01-07T08:27:25Z")

</div>

> [@Philippe\_Maincon1](#):
>
> I still have to read your last post, but my comment at this point in time is that my `@store y = 3y` should not generate a call to a function, but just leave a tag (somehow!), so that `@store function...` should do all the code generation (change to function header and calls to `@store` … in one go and with one consistent `f` -local name for function `store` .

Exactly. And that’s what the macro in my last post does. Except macro expansion goes “inwards”, i.e. `@store y = 3y` does not “leave a tag” for `@store function` to do its magic: rather, `@store` function is expanded first, and the mere presence of `@store y = 3y` _is_ the “tag”.

---

<div class="post-metadata">

**Author:** ![Philippe\_Maincon1](https://avatars.discourse-cdn.com/v4/letter/p/ec9cab/32.png) [@Philippe\_Maincon1](https://discourse.julialang.org/u/Philippe_Maincon1)\
**Post date:** [January 7, 2019, 9:01am UTC](https://discourse.julialang.org/t/metaprogramming-adding-an-arg-to-a-function/19262/11 "2019-01-07T09:01:57Z")

</div>

Your code makes sense and is very interesting to read.

A agree with your question “is this julian code”. The code relies on an assumption on Julia’s behaviour: that the outer macro call is expanded/allowed to generate first. Of course one can work around that - using a special assignment operator `y := ...` to require storage.

Merci beaucoup!