I am a bit confused by using macros to generate expressions. I would like to have a macro to make NamedTuples as follows:
a = 1
b = 2
@make_nt a b
which should return an expression that evaluates to NamedTuple{(:a, :b)}((a, b)) or (a=1, b=2)
however
macro make_nt_try(args...)
return :(NamedTuple{$args}($args))
end
returns (a = :a, b = :b)
and
macro make_nt_expr(args...)
return Expr(:(NamedTuple{$args}), args)
end
raises TypeError: in Expr, expected Symbol, got Expr
and
macro make_nt(args...)
return :(NamedTuple{$args}(tuple($(args...))))
end
works outside of functions, but not for variables in the local scope of functions.
I have read the metaprogramming section of the documentation, but did not understand when exactly evaluation happens and why it does not happen within the local scope.
Thanks for providing the answer for the question that I’m also thinking about now.
I’m very curious about the two operations ... and tuple here.
It seems like that
macro make_nt(args...)
:( $(esc.(args)) )
end
doesn’t work, as shown below
julia> @make_nt(a, b, c)
(:($(Expr(:escape, :a))), :($(Expr(:escape, :b))), :($(Expr(:escape, :c))))
(Why)?. So the ... is necessary. I think the action first ... then use tuple to collect makes sense, but why can/should we use a $in between?
Can someone teach me about this? And moreover, if this macro can be rewritten with NamedTuple(k => v for (k, v) = zip(someK, someV)) style? Thanks!
Edit: seems it’s the only available option Metaprogramming · The Julia Language.
The tuple is like a function call, being necessary to contain ....
Anyway, a macro is very similar to a function, indeed, internally it’s treated almost exactly like a function. The difference is that it runs right after parsing, before the lowering, so its input is symbolic, with Expr, Symbol, number and string literals, but no type information. So in the
@make_nt(a, b, c) # or the equivalent @make_nt a b c
the args will be (:a, :b, :c) (an NTuple{3,Symbol}). The output should also be symbolic, so we can be lazy and make the Expr corresponding to (a=a, b=b, c=c):
julia> Meta.@dump (a=a, b=b, c=c)
Expr
head: Symbol tuple
args: Array{Any}((3,))
1: Expr
head: Symbol =
args: Array{Any}((2,))
1: Symbol a
2: Symbol a
2: Expr
head: Symbol =
args: Array{Any}((2,))
1: Symbol b
2: Symbol b
3: Expr
head: Symbol =
args: Array{Any}((2,))
1: Symbol c
2: Symbol c
So, let’s make the macro by crafting together an Expr of the same form:
macro make_nt(args...)
ex = Expr(:tuple)
for a in args
push!(ex.args, Expr(:(=), a, a))
end
return ex
end
julia> @macroexpand @make_nt a b c
:((a = Main.a, b = Main.b, c = Main.c))
Note that the macro hygiene has added Main.. In general it prepends the module in which the macro is defined. Just like in a function which looks up undefined variables as a global in the module. To avoid this, to operate on the caller’s names, the Symbols must be escaped:
macro make_nt(args...)
ex = Expr(:tuple)
for a in args
push!(ex.args, Expr(:(=), a, esc(a)))
end
return ex
end
julia> @macroexpand @make_nt a b c
:((a = a, b = b, c = c))
Now, making Exprs by hand in this way is cumbersome, and so is parsing a complicated input like a for loop or other expressions. The latter is easier with the matching facilities in MacroTools.jl.
Creating expressions is easier with quote, which creates expressions from ordinary julia code, with things interpolated in with $:
macro make_nt(args...)
quote
NamedTuple{$args}($(args...))
end
end
julia> @macroexpand @make_nt a b c
quote
#= REPL[53]:3 =#
Main.NamedTuple{(:a, :b, :c)}(Main.a, Main.b, Main.c)
end
We see that the same things happens, they are prefixed by Main.. Each one of them must be escaped:
macro make_nt(args...)
quote
NamedTuple{$args}($(esc.(args)...))
end
end
julia> @macroexpand @make_nt a b c
quote
#= REPL[84]:3 =#
Main.NamedTuple{(:a, :b, :c)}(a, b, c)
end
In short, the hygiene rules, which act on the Expr returned from the macro, are made to make the Expr roughly like a function scope. Variables which are created get a unique name so they won’t interfere with anything in the scope where the macro is called, variables which are only read is prepended with the module name where the macro is defined. If you need to use the caller’s variable names, they must be esced.
You can see them in action here:
julia> macro make_nt(args...)
ex = quote
NamedTuple{$args}(($(args...),))
end
display(ex)
return ex
end
@make_nt (macro with 1 method)
julia> @macroexpand @make_nt a b c
quote
#= REPL[19]:3 =#
NamedTuple{(:a, :b, :c)}((a, b, c))
end
quote
#= REPL[19]:3 =#
Main.NamedTuple{(:a, :b, :c)}((Main.a, Main.b, Main.c))
end
The first quote is from the display inside the macro. The second is the returned expression which has been sanitized by the hygiene rules.
macro make_nt(args...)
quote
NamedTuple{$args}(tuple($(esc.(args)...)))
end
end
julia> @macroexpand @make_nt a b c
quote
#= REPL[91]:3 =#
Main.NamedTuple{(:a, :b, :c)}(Main.tuple(a, b, c))
end
or
macro make_nt(args...)
quote
NamedTuple{$args}(($(esc.(args)...),))
end
end
@make_nt (macro with 1 method)
julia> @macroexpand @make_nt a b c
quote
#= REPL[207]:3 =#
Main.NamedTuple{(:a, :b, :c)}((a, b, c))
end
julia> e = esc(:a)
:($(Expr(:escape, :a)))
julia> dump(e)
Expr
head: Symbol escape
args: Array{Any}((1,))
1: Symbol a
julia> e = Expr(:escape, :a)
:($(Expr(:escape, :a)))
julia> eval(e)
ERROR: syntax: "esc(...)" used outside of macro expansion
But the lowering/compilation step won’t accept such an Expr. The macro hygiene stuff which is run on the returns from macros will remove it.
Many of the “magic” system macros do something similar, it inserts some Expr which is meant for the lowering step:
julia> @macroexpand @inbounds begin; something; end
quote
$(Expr(:inbounds, true))
local var"#64#val" = begin
#= REPL[204]:1 =#
something
end
$(Expr(:inbounds, :pop))
var"#64#val"
end
Btw, Walter,
Here’s an interesting one for you who pondered scopes in loops:
julia> macro myintvec(N)
N isa Integer || error("argument must be a literal integer")
v = fill(0, N)
return v
end
@myintvec (macro with 1 method)
julia> for i in 1:5
v = @myintvec(5)
v[i] = i
println(v)
end
[1, 0, 0, 0, 0]
[1, 2, 0, 0, 0]
[1, 2, 3, 0, 0]
[1, 2, 3, 4, 0]
[1, 2, 3, 4, 5]
There is not really a “body part” and a “return part” in a macro. A macro is an ordinary function, though its inputs are symbolic, and so is its return value. However, it’s allowed to have julia objects in Exprs, and even a single object not inside an Expr. this is how interpolation with $ works in quote:
julia> v = fill(0,3)
3-element Vector{Int64}:
0
0
0
julia> dump( :(a = $v[1]) )
Expr
head: Symbol =
args: Array{Any}((2,))
1: Symbol a
2: Expr
head: Symbol ref
args: Array{Any}((2,))
1: Array{Int64}((3,)) [0, 0, 0] # <-- this is the v vector
2: Int64 1
While escaping the entire output is possible and sometimes convenient, it can yield surprises since it also escapes all function calls:
julia> macro add(a,b)
esc(:($a + $b))
end
@add (macro with 1 method)
julia> let
x + y = x * y
@add(2,3)
end
6
This is expected as “expand into the caller’s environment”.
I guess that the variable v is still local for each i-iteration. But you introduced a bridge so that the local v at i = 1 is “passed” successfully to i = 2, 3, 4, 5. But I’m not sure…
The thing is that the macro is run (at least) once when the loop is parsed, not once in every iteration. The macro creates a vector, and returns it. I.e. inside the loop there will be something like v = <the vector created at parse time>.
If you add another loop or function which uses the macro, it will be run once there too, creating another vector.
Sure, until the @add-macro writer decides to update the @add macro:
macro add(a, b)
esc(:(sum(($a,$b))))
end
julia> let
x + y = x * y
@add(2,3)
end
5
julia> let a=1, b=2
sum = a + b
@add(sum, a) # this worked before!
end
ERROR: MethodError: objects of type Int64 are not callable
I’m not sure whether someone has considered this before. I wrote a variant macro following the spirit of this topic, to help us construct a struct instance:
struct S
c::Int
d::Int
a::Int
b::Int
end
macro lazy_instantiate(t...)
quote
iv = map(s -> findfirst(x -> x == s, $t), fieldnames(S))
v = tuple($(map(esc, t)...))
S(map(i -> v[i], iv)...)
end
end
a, b, c, d = 2, 5, 4, 7
s = S(c, d, a, b) # use function: subject to the order
s == @lazy_instantiate(a, b, c, d) && println("use macro: permute at will")
Edit: Oh my god, I asked AI, who give me an even better solution
macro lazier_instantiate()
vals = map(esc, fieldnames(S))
:( $S($(vals...)) )
end