Improving macro programming

General Usage
,
1

I’m learning macro programming and as an experiment I programmed up a macro version of map!. It works, but seems clunky with all the escapes. What would be a more elegant way of programming the macro? I can define local variables, but there seems to be a (small) performance penalty. Thanks!

macro map(f,y,x)
    quote 
        for j = 1:length($(esc(x)))
            $(esc(y))[j] = $(esc(f))($(esc(x))[j]) 
        end
    end
end


function tryme()
    x = randn(10)
    y = zeros(10)
    @map( sin, y, x )
    println(y)
end

tryme()
2

The most elegant way would be to define it as a function instead :smile:. Macros are only really for rewriting syntax, for stuff like this a function is a more powerful, because you have access to type information, can do dispatch, use it in higher order function and makes use of dot broadcasting. You also avoid problems related to scope. Only use a macro, if there’s really no way to do it in a function, because you are doing syntactic transformations.
That said, in your case, you could just call esc on the final expression instead of each variable on its own.

8 Likes
3

Thanks @simeonschaub Yes, this is for me getting familiar with macros only. What would the final expression look like? (I had tried that and kept getting errors.)

4

Here’s how I would write that macro:

macro map(f,y,x)
    out = quote 
        for j = 1:length(x)
            $y[j] = $f($x[j]) 
        end
    end
    esc(out)
end
2 Likes
5

Note that using esc like this will cause some interesting and hard-to-find bugs.

For example, the following code, using a function to implement map, works fine:

julia> function map_function(f, y, x)
         for j in 1:length(x)
           y[j] = f(x[j])
         end
       end
map_function (generic function with 1 method)

julia> function do_stuff()
         length = 3
         x = [i for i in 1:length]
         y = zeros(length)
         map_function(sin, y, x)
         @show y
       end
do_stuff (generic function with 1 method)

julia> do_stuff()
y = [0.8414709848078965, 0.9092974268256817, 0.1411200080598672]
3-element Array{Float64,1}:
 0.8414709848078965
 0.9092974268256817
 0.1411200080598672

But what happens if we use the esc()-ed macro?

julia> macro map(f,y,x)
           out = quote 
               for j = 1:length(x)
                   $y[j] = $f($x[j]) 
               end
           end
           esc(out)
       end
@map (macro with 1 method)

julia> function do_stuff()
         length = 3
         x = [i for i in 1:length]
         y = zeros(length)
         @map(sin, y, x)
         @show y
       end
do_stuff (generic function with 1 method)

julia> do_stuff()
ERROR: MethodError: objects of type Int64 are not callable
Stacktrace:
 [1] macro expansion at ./REPL[7]:3 [inlined]
 [2] do_stuff() at ./REPL[8]:5

Oops.

This is exactly why you should not blindly esc() everything returned by the macro. What’s happening here is that there is a local variable called length. By calling esc() in the macro, you are saying “every symbol returned by this macro should refer to whatever symbol is in the scope where the macro was called”. That means that when the code returned by the macro does length(x), it tries to call the local variable length as if it were a function.

Instead, you must only escape the inputs provided by the user (f, y, and x in this case). Doing so fixes the issue:

julia> macro map(f,y,x)
           quote 
               for j = 1:length($(esc(x)))
                   $(esc(y))[j] = $(esc(f))($(esc(x))[j]) 
               end
           end
       end
@map (macro with 1 method)

julia> function do_stuff()
         length = 3
         x = [i for i in 1:length]
         y = zeros(length)
         @map(sin, y, x)
         @show y
       end
do_stuff (generic function with 1 method)

julia> do_stuff()
y = [0.8414709848078965, 0.9092974268256817, 0.1411200080598672]
3-element Array{Float64,1}:
 0.8414709848078965
 0.9092974268256817
 0.1411200080598672

There’s no avoiding the fact that doing esc correctly creates some extra noise, unfortunately. Subtleties of macro hygiene are one of many reasons to prefer writing functions unless you are doing something that can only be handled by a macro.

12 Likes
6

Thanks @rdeits ok, so you’re saying that what I had was the safest thing one could do.

2 Likes
7

Yep, this is totally right. I tend to err on the side of yolo-ing it with macro hygiene.

One option @Joris_Pinkse if you want the better namespacing and cleaner syntax is just to esc the variables before you interpolate them:

macro map(f,y,x)
    ef, ey, ex = esc.((f, y, x))
    quote 
        for j = 1:length($ex)
            $ey[j] = $ef($ex[j]) 
        end
    end
end

or even

macro map(f,y,x)
    f, y, x = esc.((f, y, x))
    quote 
        for j = 1:length($x)
            $y[j] = $f($x[j]) 
        end
    end
end
10 Likes
8

That’s a neat pattern, and you can even make it fancier with macros in your macro (so you can macro while you macro…):

julia> macro escape(args...)
         Expr(:block, [:($(esc(arg)) = esc($(esc(arg)))) for arg in args]...)
       end
@escape (macro with 1 method)

julia> macro map(f, y, x)
         @escape f x y
         quote
           for j = 1:length($x)
             $y[j] = $f($x[j])
           end
         end
       end
@map (macro with 1 method)
8 Likes
9

Thanks @Mason What’s the advantage of escaping over defining local variables inside the macro?

10

While escaping individual variables is the right thing, another possible solution here is to interpolate the function:

julia> macro map(f,y,x)
           quote 
               for j = 1:$length(x)
                   $y[j] = $f($x[j]) 
               end
           end |> esc
       end
3 Likes
11

I think it is worth emphasizing that this will interpolate the generic function Base.length, not the symbol. Eg

julia> ex = @macroexpand @map(sin, y, x)
quote
    #= REPL[19]:3 =#
    for j = 1:(length)(x)
        #= REPL[19]:4 =#
        y[j] = sin(x[j])
    end
end

julia> ex.args[2].args[1].args[2].args[3].args[1] === length
true

Now in this case this is not problematic because length is in Base. But consider

macro map(f,y,x)
    quote
        for j = 1:$mylen(x)
            $y[j] = $f($x[j])
        end
    end |> esc
end

function do_stuff()
    x = [i for i in 1:5]
    y = zeros(5)
    @map(sin, y, x)
    @show y
end

mylen(x) = length(x) # ORDER MATTERS

This errors because mylen is not defined at macroexpansion time, but works fine with the solutions that escape everything carefully (the example is contrived, but could easily happen, especially with functions scattered around in modules). Because of this, and the fact that it is easy to forget interpolating something in more complex code, I would advocate those solutions.

Macros in Julia have a lot of tricky corner cases. Despite loving them in Common Lisp, I am still trying to avoid Julia macros whenever I can. Fortunately, Julia is so powerful in other ways that this is easy to do.

3 Likes