# Improving macro programming

I’m learning macro programming and as an experiment I programmed up a macro version of map!. It works, but seems clunky with all the escapes. What would be a more elegant way of programming the macro? I can define local variables, but there seems to be a (small) performance penalty. Thanks!

``````macro map(f,y,x)
quote
for j = 1:length(\$(esc(x)))
\$(esc(y))[j] = \$(esc(f))(\$(esc(x))[j])
end
end
end

function tryme()
x = randn(10)
y = zeros(10)
@map( sin, y, x )
println(y)
end

tryme()``````

The most elegant way would be to define it as a function instead . Macros are only really for rewriting syntax, for stuff like this a function is a more powerful, because you have access to type information, can do dispatch, use it in higher order function and makes use of dot broadcasting. You also avoid problems related to scope. Only use a macro, if there’s really no way to do it in a function, because you are doing syntactic transformations.
That said, in your case, you could just call `esc` on the final expression instead of each variable on its own.

8 Likes

Thanks @simeonschaub Yes, this is for me getting familiar with macros only. What would the final expression look like? (I had tried that and kept getting errors.)

Here’s how I would write that macro:

``````macro map(f,y,x)
out = quote
for j = 1:length(x)
\$y[j] = \$f(\$x[j])
end
end
esc(out)
end
``````
2 Likes

Note that using `esc` like this will cause some interesting and hard-to-find bugs.

For example, the following code, using a function to implement `map`, works fine:

``````julia> function map_function(f, y, x)
for j in 1:length(x)
y[j] = f(x[j])
end
end
map_function (generic function with 1 method)

julia> function do_stuff()
length = 3
x = [i for i in 1:length]
y = zeros(length)
map_function(sin, y, x)
@show y
end
do_stuff (generic function with 1 method)

julia> do_stuff()
y = [0.8414709848078965, 0.9092974268256817, 0.1411200080598672]
3-element Array{Float64,1}:
0.8414709848078965
0.9092974268256817
0.1411200080598672
``````

But what happens if we use the `esc()`-ed macro?

``````julia> macro map(f,y,x)
out = quote
for j = 1:length(x)
\$y[j] = \$f(\$x[j])
end
end
esc(out)
end
@map (macro with 1 method)

julia> function do_stuff()
length = 3
x = [i for i in 1:length]
y = zeros(length)
@map(sin, y, x)
@show y
end
do_stuff (generic function with 1 method)

julia> do_stuff()
ERROR: MethodError: objects of type Int64 are not callable
Stacktrace:
 macro expansion at ./REPL:3 [inlined]
 do_stuff() at ./REPL:5
``````

Oops.

This is exactly why you should not blindly `esc()` everything returned by the macro. What’s happening here is that there is a local variable called `length`. By calling `esc()` in the macro, you are saying “every symbol returned by this macro should refer to whatever symbol is in the scope where the macro was called”. That means that when the code returned by the macro does `length(x)`, it tries to call the local variable `length` as if it were a function.

Instead, you must only escape the inputs provided by the user (`f`, `y`, and `x` in this case). Doing so fixes the issue:

``````julia> macro map(f,y,x)
quote
for j = 1:length(\$(esc(x)))
\$(esc(y))[j] = \$(esc(f))(\$(esc(x))[j])
end
end
end
@map (macro with 1 method)

julia> function do_stuff()
length = 3
x = [i for i in 1:length]
y = zeros(length)
@map(sin, y, x)
@show y
end
do_stuff (generic function with 1 method)

julia> do_stuff()
y = [0.8414709848078965, 0.9092974268256817, 0.1411200080598672]
3-element Array{Float64,1}:
0.8414709848078965
0.9092974268256817
0.1411200080598672
``````

There’s no avoiding the fact that doing `esc` correctly creates some extra noise, unfortunately. Subtleties of macro hygiene are one of many reasons to prefer writing functions unless you are doing something that can only be handled by a macro.

12 Likes

Thanks @rdeits ok, so you’re saying that what I had was the safest thing one could do.

2 Likes

Yep, this is totally right. I tend to err on the side of yolo-ing it with macro hygiene.

One option @Joris_Pinkse if you want the better namespacing and cleaner syntax is just to esc the variables before you interpolate them:

``````macro map(f,y,x)
ef, ey, ex = esc.((f, y, x))
quote
for j = 1:length(\$ex)
\$ey[j] = \$ef(\$ex[j])
end
end
end
``````

or even

``````macro map(f,y,x)
f, y, x = esc.((f, y, x))
quote
for j = 1:length(\$x)
\$y[j] = \$f(\$x[j])
end
end
end
``````
10 Likes

That’s a neat pattern, and you can even make it fancier with macros in your macro (so you can macro while you macro…):

``````julia> macro escape(args...)
Expr(:block, [:(\$(esc(arg)) = esc(\$(esc(arg)))) for arg in args]...)
end
@escape (macro with 1 method)

julia> macro map(f, y, x)
@escape f x y
quote
for j = 1:length(\$x)
\$y[j] = \$f(\$x[j])
end
end
end
@map (macro with 1 method)
``````
8 Likes

Thanks @Mason What’s the advantage of escaping over defining local variables inside the macro?

While escaping individual variables is the right thing, another possible solution here is to interpolate the function:

``````julia> macro map(f,y,x)
quote
for j = 1:\$length(x)
\$y[j] = \$f(\$x[j])
end
end |> esc
end
``````
3 Likes

I think it is worth emphasizing that this will interpolate the generic function `Base.length`, not the symbol. Eg

``````julia> ex = @macroexpand @map(sin, y, x)
quote
#= REPL:3 =#
for j = 1:(length)(x)
#= REPL:4 =#
y[j] = sin(x[j])
end
end

julia> ex.args.args.args.args.args === length
true
``````

Now in this case this is not problematic because `length` is in `Base`. But consider

``````macro map(f,y,x)
quote
for j = 1:\$mylen(x)
\$y[j] = \$f(\$x[j])
end
end |> esc
end

function do_stuff()
x = [i for i in 1:5]
y = zeros(5)
@map(sin, y, x)
@show y
end

mylen(x) = length(x) # ORDER MATTERS
``````

This errors because `mylen` is not defined at macroexpansion time, but works fine with the solutions that escape everything carefully (the example is contrived, but could easily happen, especially with functions scattered around in modules). Because of this, and the fact that it is easy to forget interpolating something in more complex code, I would advocate those solutions.

Macros in Julia have a lot of tricky corner cases. Despite loving them in Common Lisp, I am still trying to avoid Julia macros whenever I can. Fortunately, Julia is so powerful in other ways that this is easy to do.

3 Likes