Calling a macro from within a macro, revisited



Hi all,

I have a question: am i seeing a bug in the macro expander, or am I misunderstanding its usage?

I’m implementing a macro which delegates (most of) its work to another macro, and I’m running into some issues around escaping. Here is a minimal version which I think should work, based on the discussion in this related issue, but isn’t:

module Retest

using Test

macro retestset1(args...)
    :(@testset($((esc(a) for a in args)...)))  # should work?

macro retestset2(args...)
    esc(:($Test.@testset($(args...))))         # works but eww?


Note that this is a completely generic problem—the fact that @testset is the macro being delegated to is just a detail, and we should be able to “wrap” any macro we want without changes to that macro’s implementation.

Here’s the error trying to invoke @retesetse1:

julia> Retest.@retestset1 begin end
ERROR: LoadError: Expected begin/end block or for loop as argument to @testset
 [1] @testset(::LineNumberNode, ::Module, ::Vararg{Any,N} where N) at /Users/osx/buildbot/slave/package_osx64/build/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1041
in expression starting at /Volumes/GoogleDrive/My Drive/Retest/src/Retest.jl:6

A little investigation indicates that the issue is that the expressions passed to @testset still contain escape nodes, which it chokes on. Here’s an even simpler example illustrating the phenomenon:

julia> macro foo(arg)
           @show arg
@foo (macro with 1 method)

julia> macro bar(arg)
@bar (macro with 1 method)

julia> @bar 1+1
arg = :($(Expr(:escape, :(1 + 1))))

Is this how the macroexpander is supposed to operate? If so, how do I reconcile this with the advice from @Tamas_Papp in Call a macro within a macro?

Invoking @retestset2 on the other hand works fine, including picking up variables from the calling environment:

julia> let x=1
       Retest.@retestset2 begin @assert x == 1 end
Test Summary: |
test set      | No tests
Test.DefaultTestSet("test set", Any[], 0, false)

But it seems like a bit of a kludge. Any help appreciated.


I suggest writing the macros as just wrappers around functions (that take an expression as argument and return an expression), and calling the respective functions instead.


In general, I completely agree. In this case, though, the OP’s goal was to call @testset; is there a good function equivalent to that?

Either way, I think as OP says, the escape nodes should’ve been removed by the macro expander before calling the macro returned in the quoted code. Seeing :($(Expr(:escape, :(1 + 1))) being provided as the input to @foo() does seem like it’s not behaving as expected, and might be a bug in the macro expander?


Thanks @dpsanders, but I don’t quite follow, can you show me a working example of what you mean? Also, can anyone comment on the @foo/@bar behavior, is that working as expected?


@NHDaly is right that I spoke a bit too fast, since you’re wrapping @testset.

But in my own code, I would define something like

julia> macro hello(ex)
           return hello(ex)
@hello (macro with 1 method)

julia> hello(ex) = ex
hello (generic function with 1 method)

julia> @hello 3+4

Here the macro exists just to pick up what the user types into an expression, and pass this expression over to a function that actually does the hard work.

If this is the case then you can just call the corresponding function instead inside your other macro.
The case of @testset seems to be a bit harder, though.


esc please…


(@yuyichao, do you mean @dpsanders’s example should include esc? The OP, @tjgreen,'s examples all have appropriate escaping, I think?)


Correct. ( I thought I pressed the button to reply to that post) edit: hmmm seems that it might not be working in my phone.

Also, my point is that esc is the whole reason nesting is a problem at all so in additional to the normal reason of not putting out misleading info it s especially important for this issue to include it.


Right, my situation is different, and I’m really asking a more general question: given a macro @f in some other library, how do you write a macro @g which behaves exactly like @f (and is implemented in terms of @f)? And, if my first attempt at this for the case of @testset (@retestset1) isn’t working, is this because of a bug in the macroexpander, or because I wrote it wrong, or something else?


What you want is to define it like this

macro retestset1(arg)


julia> @retestset1 begin
           @test 1 == 1.0
Test Summary: | Pass  Total
test set      |    1      1
Test.DefaultTestSet("test set", Any[], 1, false)

Your issue is that you are not using the Julia abstract syntax tree correctly.


@chakravala Hmm, so this works, but only for one form of the arguments to @testset (which can also take a for-loop instead of a block, and also has an optional name parameter). Can you think of a generic implementation that doesn’t make any assumptions about the shape of the underlying macro’s arguments?



macro retestset1(arg)

simply replace :block with arg.head


There is no solution to this problem, it’s still an open issue:

IIRC the only completely general workaround is for your macros to understand that they may see Expr(:escape) and in some cases to partially understand lowering :frowning:

See also


Awesome, thanks for the authoritative answer @Chris_Foster!