Escaping of nested blocks

General Usage
1

Hi,

I am trying to learn to work with macros and ASTs, but am running into a bit of confusion regarding escaping. I haven’t found a lot of documentation regarding this, but if there is an authorative source, I would appreciate any pointers.

I wrote some example code which looks similar to this:

function op_helper(ee::Expr)
...
end

macro op(ee::Expr)
   $(op_helper(ee))
end

Inside of op_helper, I recursively walk the AST and do some very simple transformations (I am effectively trying to just return identity) on the expressions.

I am running into an interesting conundrum, which is challenging my understanding of esc in recursive contexts.

Consider the following test function:

function test(a)
    x = @op  a + 1 #1
    @op y = a + 2 #2

    @op begin  #3
         z = a + 3 #4
    end
end

I am finding the following:

  1. is straightforward, and my code simply recursively traverses the AST, only escaping
# ...
if typeof(ee) == Symbol 
   esc(ee)
elseif #...
  1. Seems only a little bit more work is required (note for simplicity, I’m not recursing on the LHS)
if typeof(ee) == Expr && ee.head == :(=)
    esc(:($(ee.args[1]) = $(op_helper(ee.args[2])))
elseif #...
  1. It is this case which is intriguing.
if typeof(ee) == Expr && ee.head == :block
    parts = [op_helper(elem) for elem in ee.args]
    Expr(:block, parts...) #5
elseif #...

It appears that for this to work, I actually need to write esc(Expr(:block, parts...)) in #5; however, this then causes problems with #4, which now doesn’t want to be escaped, unlike the recursive code path for #2.

Is what I’m describing correct, or am I missing something?

Thank you,

Tom

2

This looks wrong. I assume you mean op_helper(ee).

You must escape the LHS.

What do you mean by not working?

3

Yes. Typed the wrong thing, sorry.

I think it is escaped; I just said I didn’t recurse.

The compiler complains:

ERROR: LoadError: syntax: unhandled expr

Anyway, I think I’ve figured it out. I just generate the AST using op_helper(), and then just esc’ape the resulting expression in op.

Thanks for the help,

Tom

4

At least the code you paste didn’t.

Do note that this turns off macro hygiene and you can have accidental reference to user local variable that’s not intended.

5

Hi Yuyichao,

Thank you for your time. I was wondering if you could clarify your comment for me, because I’m afraid I may not be understanding something correctly.

I wrote

if typeof(ee) == Expr && ee.head == :(=)
    esc(:($(ee.args[1]) = $(op_helper(ee.args[2])))
elseif #...

Say, in the case of #2, I would have expected this to expand to something like

esc(:(y = a + 2))

You assert that the code is not escaped. Would you be able to explain? I’ve copied the entire example code below for reference:

function op_helper(ee::Expr)
    if typeof(ee) == Symbol 
       esc(ee)
    elseif typeof(ee) == Expr && ee.head == :(=)
        esc(:($(ee.args[1]) = $(op_helper(ee.args[2]))))
    elseif typeof(ee) == Expr && ee.head == :block
        parts = [op_helper(elem) for elem in ee.args]
        Expr(:block, parts...) #5
    elseif typeof(ee) == Expr && ee.head == :call
        ee
    elseif ee.head == :line
        # Nothing
    else
        error("Not handled", ee)
    end
end

macro op(ee::Expr)
    res = op_helper(ee)
    println(res)
    res
end

function test(a)
    x = @op a + 1 #1
    @op y = a + 2 #2

    @op begin  #3
         z = a + 3 #4
    end

    return (x, y, z)
end

println(test(3))

Yes, that was partially my worry. It appears that my new code is able to avoid it to some degree, at least.

Thanks,

Tom

6

I missed the esc around the whole thing. That does escape the LHS but is wrong. In general, you must escape each piece of user input once and exactly once. A slightly better implementation would be

julia> function op_helper(ee::ANY)
           if isa(ee, Symbol) 
               esc(ee)
           elseif isa(ee, Expr) && ee.head == :(=)
               :($(esc(ee.args[1])) = $(op_helper(ee.args[2])))
           elseif isa(ee, Expr) && ee.head == :block
               parts = [op_helper(elem) for elem in ee.args]
               Expr(:block, parts...) #5
           elseif isa(ee, Expr) && ee.head == :call
               esc(ee)
           elseif isa(ee, Expr) && ee.head == :line
               esc(ee)
           else
               error("Not handled", ee)
           end
       end
op_helper (generic function with 1 method)

julia> macro op(ee::Expr)
           res = op_helper(ee)
           println(res)
           res
       end
@op (macro with 1 method)

julia> function test(a)
           x = @op a + 1 #1
           @op y = a + 2 #2

           @op begin  #3
                z = a + 3 #4
           end

           return (x, y, z)
       end
$(Expr(:escape, :(a + 1)))
$(Expr(:escape, :y)) = $(Expr(:escape, :(a + 2)))
begin 
    $(Expr(:escape, :( # REPL[3], line 6:)))
    $(Expr(:escape, :z)) = $(Expr(:escape, :(a + 3)))
end
test (generic function with 1 method)

julia> println(test(3))
(4,5,6)

Note that handling Symbols but not other literals is generally problematic. (although actually neither is handled in either version)

7

Hi yuyichao,

Thank you very much for the example code; that helped a lot. I actually had something almost identical to what you wrote to begin with, but couldn’t get it to work. The main difference was my initial attempt of the assignment, which I couldn’t get working

:(esc($(ee.args[1])) = $(op_helper(ee.args[2])))

versus your version

:($(esc(ee.args[1])) = $(op_helper(ee.args[2])))

This was actually the reason why I wrapped the whole expression in esc, instead of just the destination.

Let me see if I can my code going now.

Thank you for your help, I really appreciate it.

Kind regards,

Tom

8

I wouldn’t disagree that expression interpolation can easily give you multiple levels of parenthesis. At least you can always assign it to a variable (e.g. earg1 = esc(ee.args[1])) and in this sense it’s not too different from an expression with nested function calls.

9

Hi yuyichao,

I wasn’t really referring the parenthesis, but more to the

$(esc(<expr>)) vs esc($(<expr>)).

I can’t say I fully see the difference (other than one working and the other not).

Thanks,

Tom

10

I know. I’m just saying that the many level of parentheses makes it harder to tell the difference.

$ is the interpolation so in $(esc(<expr>)) it interpolated an escaped expression (what you want) and in esc($(<expr>)) it interpolate the original expression into a function call of esc.