working on an ecological model, I have a macro @species that creates a new struct using @kwdef. In most use cases, this works just fine. However, if I use this macro outside of the module in which it was defined, the keyword constructor doesn’t work. Here’s a MWE:
### species module
module CoreModule
export @species, Fox
macro species(name, body)
quote
@kwdef mutable struct $(name)
const id::Int
$(body.args...)
end
$(esc(name))(id) = $(esc(name))(id=id)
end
end
@species Fox begin
cleverness::Float64=1.0
end
# <-- everything works inside the core module -->
println(Fox(5, 10)) # use the default positional constructor
println(Fox(id=5, cleverness=10)) # use the keyword constructor
println(Fox(5)) # use the outer constructor
end
### top level
using .CoreModule
# <-- calling species defined inside the core module also works outside -->
println(Fox(5, 10)) # use the default positional constructor
println(Fox(id=5, cleverness=10)) # use the keyword constructor
println(Fox(5)) # use the outer constructor
@species Rabbit begin
speed::Float64=2.0
end
# <-- but calling a species defined outside the core module using its keyword
# constructors (i.e. the last two calls below) gives a MethodError -->
println(Rabbit(42, 12)) # use the default positional constructor
println(Rabbit(id=42, speed=12.0)) # use the keyword constructor
println(Rabbit(42)) # use the outer constructor
julia> @macroexpand @species Rabbit begin
speed::Float64=2.0
end
quote
#= REPL[17]:6 =#
begin
#= util.jl:609 =#
begin
$(Expr(:meta, :doc))
mutable struct Rabbit
#= REPL[17]:7 =#
const id::Main.CoreModule.Int
#= REPL[17]:8 =#
#= REPL[24]:2 =#
speed::Main.CoreModule.Float64
end
end
#= util.jl:610 =#
function Main.CoreModule.Rabbit(; id, speed = 2.0)
#= REPL[17]:6 =#
Main.CoreModule.Rabbit(id, speed)
end
end
#= REPL[17]:10 =#
Rabbit(var"#7#id") = begin
#= REPL[17]:10 =#
Rabbit(id = var"#7#id")
end
end
we can see near the end that your esc works for the method in the quote, your struct would belong to the macro call scope as named, but all the argument annotations and methods made by @kwdef belong to Main.CoreModule, the species macro definition scope. Contrary to your intention, it seems @kwdef (which uses a fair bit of esc itself) considers CoreModule its call scope. While true that macro calls don’t get expanded in quotes, we don’t control where it is expanded between returning the quote and the evaluation in the destination. You could write const id::$(esc(:Int)) to “fix” CoreModule’s side of the hygiene but that’s not the issue. EDIT: I did a test where I changed kwdef to println("kwdef: ", __module__) to show where it expands, and it does report Main. However, a single esc still refers to the scope where the call is written. This effect is independent of what @kwdef does, as in the example below. 2 esc would do the trick, but altering the borrowed @kwdef to do the exact right number of esc for quote nesting is not a good approach.
julia> module A # substitute for CoreModule
module B # subsitute for Base
macro bb() # substitute for @kwdef
println("bb: ", __module__, __source__)
quote
struct X x::$(esc(:Int)) end # esc(:X) is invalid
$(esc(:X))() = $(esc(:X))(1)
end
end
end
macro aa() # substitute for @species
println("aa: ", __module__, __source__)
:(B.@bb)
end
end
Main.A
julia> @macroexpand A.@aa # esc refers to Main.A, not Main
aa: Main#= REPL[47]:1 =#
bb: Main#= REPL[45]:13 =#
quote
#= REPL[45]:6 =#
struct X
#= REPL[45]:6 =#
x::Main.A.Int
end
#= REPL[45]:7 =#
Main.A.X() = begin
#= REPL[45]:7 =#
Main.A.X(1)
end
end
julia> module A
module B
macro bb()
println("bb: ", __module__, __source__)
quote
struct X x::$(esc(esc(:Int))) end
$(esc(esc(:X)))() = $(esc(esc(:X)))(1)
end
end
end
macro aa()
println("aa: ", __module__, __source__)
:(B.@bb)
end
end
WARNING: replacing module A.
Main.A
julia> @macroexpand A.@aa # esc(esc makes it to Main
aa: Main#= REPL[80]:1 =#
bb: Main#= REPL[79]:13 =#
quote
#= REPL[79]:6 =#
struct X
#= REPL[79]:6 =#
x::Int
end
#= REPL[79]:7 =#
X() = begin
#= REPL[79]:7 =#
X(1)
end
end
Unfortunately I don’t have much experience writing macros, hopefully someone else has an idea on how to deal with macro calls nested in quotes. My hunch is the least effort is to call @species @kwdef struct ... so both macro calls are written at the destination; species expands first to alter the struct part of the @kwdef expression before kwdef expands to make the methods.
The problem, which can be seen by looking at the macroexpansion with
@macroexpand @species Rabbit begin
speed::Float64=2.0
end
is that the @kwdef is run in the CoreModule context, so that the keyword constructor is defined inside CoreModule, and not directly available from the outside. Write your macro as:
macro species(name, body)
esc(quote
@kwdef mutable struct $(name)
const id::Int
$(body.args...)
end
$(name)(id) = $(name)(id=id)
end)
end
I.e. escape the entire quote, and no esc inside it.
Makes a lot of sense to esc the expression that contains @kwdef (I suppose that also counts as a variable for hygiene?), I can confirm this works for @species. Trying it on A.aa, it seemed that I need to qualify the macro call how it would be accessed in the expansion scope (like esc(:(A.B.@bb)), which isn’t a problem for the Base-exported @kwdef); A.B.bb still has to esc its own output as well. I worked around it by interpolating the module instance into the expression esc(:($B.@bb)) (which is not $(B.@bb) which would expand before interpolating).