# Calling a macro from within a macro, revisited

**URL:** <https://discourse.julialang.org/t/calling-a-macro-from-within-a-macro-revisited/19680>\
**Category:** General Usage\
**Tags:** question\
**Created:** [January 15, 2019, 10:27pm UTC](https://discourse.julialang.org/t/calling-a-macro-from-within-a-macro-revisited/19680 "2019-01-15T22:27:01Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![tjgreen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tjgreen/32/6749_2.png) [@tjgreen](https://discourse.julialang.org/u/tjgreen)\
**Post date:** [January 15, 2019, 10:27pm UTC](https://discourse.julialang.org/t/calling-a-macro-from-within-a-macro-revisited/19680/1 "2019-01-15T22:27:01Z")

</div>

Hi all,

I have a question: am i seeing a bug in the macro expander, or am I misunderstanding its usage?

I’m implementing a macro which delegates (most of) its work to another macro, and I’m running into some issues around escaping. Here is a minimal version which I think should work, based on the discussion in this [related issue](https://discourse.julialang.org/t/call-a-macro-within-a-macro/3708), but isn’t:

```julia
module Retest

using Test

macro retestset1(args...)
    :(@testset($((esc(a) for a in args)...))) # should work?
end

macro retestset2(args...)
    esc(:($Test.@testset($(args...)))) # works but eww?
end

end

```

Note that this is a completely generic problem—the fact that `@testset` is the macro being delegated to is just a detail, and we should be able to “wrap” any macro we want without changes to that macro’s implementation.

Here’s the error trying to invoke `@retesetse1`:

```julia
julia> Retest.@retestset1 begin end
ERROR: LoadError: Expected begin/end block or for loop as argument to @testset
Stacktrace:
 [1] @testset(::LineNumberNode, ::Module, ::Vararg{Any,N} where N) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1041
in expression starting at /Volumes/GoogleDrive/My Drive/Retest/src/Retest.jl:6

```

A little investigation indicates that the issue is that the expressions passed to `@testset` still contain escape nodes, which it chokes on. Here’s an even simpler example illustrating the phenomenon:

```julia
julia> macro foo(arg)
           @show arg
       end
@foo (macro with 1 method)

julia> macro bar(arg)
           :(@foo($(esc(arg))))
       end
@bar (macro with 1 method)

julia> @bar 1+1
arg = :($(Expr(:escape, :(1 + 1))))
2

```

Is this how the macroexpander is supposed to operate? If so, how do I reconcile this with the advice from @Tamas_Papp in [Call a macro within a macro - #6 by findmyway](https://discourse.julialang.org/t/call-a-macro-within-a-macro/3708/6)?

Invoking `@retestset2` on the other hand works fine, including picking up variables from the calling environment:

```julia
julia> let x=1
       Retest.@retestset2 begin @assert x == 1 end
       end
Test Summary: |
test set | No tests
Test.DefaultTestSet("test set", Any[], 0, false)

```

But it seems like a bit of a kludge. Any help appreciated.

---

<div class="post-metadata">

**Author:** ![dpsanders](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dpsanders/32/3573_2.png) [@dpsanders](https://discourse.julialang.org/u/dpsanders)\
**Post date:** [January 15, 2019, 10:27pm UTC](https://discourse.julialang.org/t/calling-a-macro-from-within-a-macro-revisited/19680/2 "2019-01-15T22:27:40Z")

</div>

I suggest writing the macros as just wrappers around functions (that take an expression as argument and return an expression), and calling the respective functions instead.

---

<div class="post-metadata">

**Author:** ![NHDaly](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nhdaly/32/1413_2.png) [@NHDaly](https://discourse.julialang.org/u/NHDaly)\
**Post date:** [January 15, 2019, 11:00pm UTC](https://discourse.julialang.org/t/calling-a-macro-from-within-a-macro-revisited/19680/3 "2019-01-15T23:00:48Z")

</div>

In general, I completely agree. In this case, though, the OP’s goal was to call `@testset`; is there a good function equivalent to that?

Either way, I think as OP says, the `escape` nodes should’ve been removed by the macro expander before calling the macro returned in the quoted code. Seeing `:($(Expr(:escape, :(1 + 1)))` being provided as the input to `@foo()` does seem like it’s not behaving as expected, and might be a bug in the macro expander?

---

<div class="post-metadata">

**Author:** ![tjgreen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tjgreen/32/6749_2.png) [@tjgreen](https://discourse.julialang.org/u/tjgreen)\
**Post date:** [January 15, 2019, 11:04pm UTC](https://discourse.julialang.org/t/calling-a-macro-from-within-a-macro-revisited/19680/4 "2019-01-15T23:04:38Z")

</div>

Thanks @dpsanders, but I don’t quite follow, can you show me a working example of what you mean? Also, can anyone comment on the `@foo`/`@bar` behavior, is that working as expected?

---

<div class="post-metadata">

**Author:** ![dpsanders](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dpsanders/32/3573_2.png) [@dpsanders](https://discourse.julialang.org/u/dpsanders)\
**Post date:** [January 15, 2019, 11:10pm UTC](https://discourse.julialang.org/t/calling-a-macro-from-within-a-macro-revisited/19680/5 "2019-01-15T23:10:35Z")

</div>

@NHDaly is right that I spoke a bit too fast, since you’re wrapping `@testset`.

But in my own code, I would define something like

```julia
julia> macro hello(ex)
           return hello(ex)
       end
@hello (macro with 1 method)

julia> hello(ex) = ex
hello (generic function with 1 method)

julia> @hello 3+4
7

```

Here the macro exists just to pick up what the user types into an expression, and pass this expression over to a function that actually does the hard work.

If this is the case then you can just call the corresponding function instead inside your other macro.  
The case of `@testset` seems to be a bit harder, though.

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [January 15, 2019, 11:17pm UTC](https://discourse.julialang.org/t/calling-a-macro-from-within-a-macro-revisited/19680/6 "2019-01-15T23:17:32Z")

</div>

esc please…

---

<div class="post-metadata">

**Author:** ![NHDaly](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nhdaly/32/1413_2.png) [@NHDaly](https://discourse.julialang.org/u/NHDaly)\
**Post date:** [January 15, 2019, 11:21pm UTC](https://discourse.julialang.org/t/calling-a-macro-from-within-a-macro-revisited/19680/7 "2019-01-15T23:21:27Z")

</div>

(@yuyichao, do you mean @dpsanders’s example should include `esc`? The OP, @tjgreen,'s examples all have appropriate escaping, I think?)

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [January 15, 2019, 11:28pm UTC](https://discourse.julialang.org/t/calling-a-macro-from-within-a-macro-revisited/19680/8 "2019-01-15T23:28:51Z")

</div>

Correct. ( I thought I pressed the button to reply to that post) edit: hmmm seems that it might not be working in my phone.

Also, my point is that esc is the whole reason nesting is a problem at all so in additional to the normal reason of not putting out misleading info it s especially important for this issue to include it.

---

<div class="post-metadata">

**Author:** ![tjgreen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tjgreen/32/6749_2.png) [@tjgreen](https://discourse.julialang.org/u/tjgreen)\
**Post date:** [January 15, 2019, 11:33pm UTC](https://discourse.julialang.org/t/calling-a-macro-from-within-a-macro-revisited/19680/9 "2019-01-15T23:33:40Z")

</div>

Right, my situation is different, and I’m really asking a more general question: given a macro `@f` in some other library, how do you write a macro `@g` which behaves exactly like `@f` (and is implemented in terms of `@f`)? And, if my first attempt at this for the case of `@testset` (`@retestset1`) isn’t working, is this because of a bug in the macroexpander, or because I wrote it wrong, or something else?

---

<div class="post-metadata">

**Author:** ![chakravala](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chakravala/32/6832_2.png) [@chakravala](https://discourse.julialang.org/u/chakravala)\
**Post date:** [January 15, 2019, 11:54pm UTC](https://discourse.julialang.org/t/calling-a-macro-from-within-a-macro-revisited/19680/10 "2019-01-15T23:54:52Z")

</div>

> [@tjgreen](#):
>
> macro retestset1(args…) ☹@testset($((esc(a) for a in args)…))) # should work? end

What you want is to define it like this

```Julia
macro retestset1(arg)
    :(@testset$(Expr(:block,esc.(arg.args)...))
end

```

then

```Julia
julia> @retestset1 begin
           @test 1 == 1.0
       end
Test Summary: | Pass Total
test set | 1 1
Test.DefaultTestSet("test set", Any[], 1, false)

```

Your issue is that you are not using the Julia abstract syntax tree correctly.

---

<div class="post-metadata">

**Author:** ![tjgreen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tjgreen/32/6749_2.png) [@tjgreen](https://discourse.julialang.org/u/tjgreen)\
**Post date:** [January 16, 2019, 12:17am UTC](https://discourse.julialang.org/t/calling-a-macro-from-within-a-macro-revisited/19680/11 "2019-01-16T00:17:44Z")

</div>

@chakravala Hmm, so this works, but only for one form of the arguments to `@testset` (which can also take a for-loop instead of a block, and also has an optional name parameter). Can you think of a generic implementation that doesn’t make any assumptions about the shape of the underlying macro’s arguments?

---

<div class="post-metadata">

**Author:** ![chakravala](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chakravala/32/6832_2.png) [@chakravala](https://discourse.julialang.org/u/chakravala)\
**Post date:** [January 16, 2019, 12:21am UTC](https://discourse.julialang.org/t/calling-a-macro-from-within-a-macro-revisited/19680/12 "2019-01-16T00:21:30Z")

</div>

> [@tjgreen](#):
>
> Can you think of a generic implementation that doesn’t make any assumptions about the shape of the underlying macro’s arguments?

Yes,

```Julia
macro retestset1(arg)
    :(@testset$(Expr(arg.head,esc.(arg.args)...))
end

```

simply replace `:block` with `arg.head`

---

<div class="post-metadata">

**Author:** ![c42f](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/c42f/32/52842_2.png) [@c42f](https://discourse.julialang.org/u/c42f)\
**Post date:** [January 16, 2019, 4:46am UTC](https://discourse.julialang.org/t/calling-a-macro-from-within-a-macro-revisited/19680/13 "2019-01-16T04:46:29Z")

</div>

There is no solution to this problem, it’s still an open issue: [macro hygine escape stripped late for macros calling macros · Issue #23221 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/issues/23221)

IIRC the only completely general workaround is for your macros to understand that they may see `Expr(:escape)` and in some cases to partially understand lowering ☹

See also [FastClosures.jl/FastClosures.jl at c4c8cf2d46024a875d6ab967b207709218015dd0 · c42f/FastClosures.jl · GitHub](https://github.com/c42f/FastClosures.jl/blob/c4c8cf2d46024a875d6ab967b207709218015dd0/src/FastClosures.jl#L138)

---

<div class="post-metadata">

**Author:** ![tjgreen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tjgreen/32/6749_2.png) [@tjgreen](https://discourse.julialang.org/u/tjgreen)\
**Post date:** [January 16, 2019, 3:49pm UTC](https://discourse.julialang.org/t/calling-a-macro-from-within-a-macro-revisited/19680/14 "2019-01-16T15:49:14Z")

</div>

Awesome, thanks for the authoritative answer @c42f!

---

<div class="post-metadata">

**Author:** ![johnomotani](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnomotani/32/26753_2.png) [@johnomotani](https://discourse.julialang.org/u/johnomotani)\
**Post date:** [July 31, 2021, 3:53pm UTC](https://discourse.julialang.org/t/calling-a-macro-from-within-a-macro-revisited/19680/15 "2021-07-31T15:53:07Z")

</div>

I know this is an old thread, but I’ve ended up back here a few times, so I think it might be useful to add on. I saw that the `@simd` macro uses an `esc()` outside the entire expression like this: [https://github.com/JuliaLang/julia/blob/1b93d53fc4bb59350ada898038ed4de2994cce33/base/simdloop.jl#L127-L129](https://github.com/JuliaLang/julia/blob/1b93d53fc4bb59350ada898038ed4de2994cce33/base/simdloop.jl#L127-L129)

Following a similar pattern seems to work for wrapping `@simd`. I have a macro

```julia
macro innerloop(ex)
    return esc( :( @simd ivdep $ex ) )
end

```

so I can change the macros applied to inner loops throughout my code. It seems to be working at first glance. Not sure if it’s a general solution, and I confess I’m fairly new to Julia and not really comfortable with how macros/metaprogramming work - should this work? Anyone have comments?

---

<div class="post-metadata">

**Author:** ![c42f](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/c42f/32/52842_2.png) [@c42f](https://discourse.julialang.org/u/c42f)\
**Post date:** [August 5, 2021, 4:07am UTC](https://discourse.julialang.org/t/calling-a-macro-from-within-a-macro-revisited/19680/16 "2021-08-05T04:07:46Z")

</div>

> [@johnomotani](#):
>
> should this work? Anyone have comments?

This “works” by disabling all macro hygiene. In this case you’re assuming that the module which uses `@innerloop ex` also has the macro `@simd` available. But this might not always be the case.

Disabling hygiene can be an acceptable workaround for macros which aren’t meant to be reused in other modules, or generally where the macro author is also the user of the macro so they can manually ensure hygiene at the use site.

But in general this isn’t a great idea because it leads to symbols being looked up in the wrong scope which can be very confusing for macro users.