Macro defining an internal macro?

General Usage
1

I need some help with macros.

I have a directed graph that is represented by a collection of Pairs where each Pair represents an arc of the graph.

I want to perform some transformations on the graph. Each transformation function will be called on each :first of each Pair, perform some tests (querying the graph in the process), and, if none of those tests fail, will return two collections of Pairs. The first collection is of arcs that should be added to the graph, and the second of arcs to be removed.

To simplify the writing of these transformation functions, I want a macro that provides three operations:

  1. check(condition): to evaluate a condition and return from the function if the condition is not met;

  2. add(arc): add an arc to the first return value (arcs to be added to the graph);

  3. remove(arc): add an arc to the second return value (arcs to be removed from the graph).

add and remove can be implemented as internal functions, but I don’t think check can since it needs to return from an outer context.

When I try to define the check macro within the macro that provides this DSL to the body of a transformation function though. I get an error::

using Markdown
import MacroTools

macro graphTransformerBody(bodyblock)
  quote
    let
      addset = Set()
      removeset = Set()
          add(x) = push!(addset, x)
          remove(x) = push!(removeset, x)

          macro check(condition)
            quote
              #=
              if ! $condition
                return addset, removeset
              end
              =#
            end
          end

          $bodyblock
      return addset, removeset
    end
  end
end


show(MacroTools.prewalk(MacroTools.rmlines, 
@macroexpand( @graphTransformerBody(let
  @check(foobar)
  add(x)
end)
        )))

produces

quote
    let
        var"#1#addset" = Main.Set()
        var"#2#removeset" = Main.Set()
        var"#3#add"(var"#9#x") = begin
                Main.push!(var"#1#addset", var"#9#x")
            end
        var"#4#remove"(var"#10#x") = begin
                Main.push!(var"#2#removeset", var"#10#x")
            end
        macro Main.check(var"#11#condition")
            $(Expr(:copyast, :($(QuoteNode(quote
    #= c:\Users\Mark Nahabedian\.julia\dev\PanelCutting\src\macrotest.jl:19 =#
end)))))
        end
        let
            var"#3#add"(Main.x)
        end
        return (var"#1#addset", var"#2#removeset")
    end
end

as I expect.

If I uncomment the invocation of '@check` thouh, I get
the error

ERROR: LoadError: LoadError: UndefVarError: @check not defined

Any suggestions for how I can get this to wokr?

Thanks.

2

I don’t know if this is the issue you’re facing but there has been some discussion that under the current design, macros defining macros are hard to use.

https://github.com/JuliaLang/julia/issues/37691

https://github.com/JuliaLang/julia/pull/6910

1 Like
3

You’re just missing $(esc(bodyblock)). A general rule of thumb is that all arguments need to be wrapped in esc, and nothing else should be. There are exceptions, but that is usually good enough place to begin.

4

Oh, sorry, I misread this as a hygiene question. It is a phasing question instead however. You are not defining an inner macro in this macro, you are defining how to define an inner macro with this macro. But that sequencing implies that the new macro won’t be available.

To do this, you’d instead write a pre-walk pass that pre-parses bodyblock, and calls a function on it.

macro graphTransformerBody(bodyblock)
    addset = Set()
    removeset = Set()
    add(x) = push!(addset, x)
    remove(x) = push!(removeset, x)
    check(x) = condition ? return (addset, removeset) : nothing
    bodyblock = preprocess!(bodyblock, check, :macrocall, sym"@check") # user implemented function taht walks over bodyblock AST and calls `check` on every Expr that matches this :macrocall
    return quote
        $(esc(bodyblock))
        return addset, removeset # N.B. the return keyword here is very unusual
    end
end
5

Thanks Jameson and jzr.

I thought walking code was what evaluators and compilers were fpr :slight_smile:

Where does preprocess! come from? I can’t find documentation for it (julia 1.6.0).

6

You might need to scroll above to read the comment: this is a function you need to implement. Yes, compilers also analyze the code, but here you are trying to intercept that to do something different, so you need to implement that something different.

7

Thanks Jameson. That comment didn’t sink in at 2am when I first read it but when I reread it yesterday morning it did.

Having to do that seems way to fraught to me. I’m giving up on using nested macros. I’m implementing check as a local function that throws to a catch handler to do its non-local exit. I’m also doing my own macro hyhiene since the regular mechanism was getting in the way. Here’s my new code:

struct NonlocalTransfer <: Exception
  uid

  function NonlocalTransfer()
    new(UUIDs.uuid1())
  end
end

function Base.showerror(io::IO, e::NonlocalTransfer)
  print(io, "NonlocalTransfer not caught!")
end

macro graphTransformerBody(bodyblock)
      exittag = gensym("exittag")
      addset = gensym("addset")
      removeset = gensym("removeset")
  esc(quote
    let
      $exittag = NonlocalTransfer()
      $addset = Set()
      $removeset = Set()
      add(x) = push!($addset, x)
      remove(x) = push!($removeset, x)
      function check(condition)
        if !condition
          throw($exittag)
        end
      end
      try
        $bodyblock
      catch e
        if e != $exittag
          rethrow(e)
        end
      end
      return $addset, $removeset
    end
      end)
end
8

Does this problem really need a macro? How about something like this:

function graph_transform(f)
    exittag = NonlocalTransfer()
    addset = Set()
    removeset = Set()
    add(x) = push!(addset, x)
    remove(x) = push!(removeset, x)
    check(condition) = condition || throw(exittag)
    try
        f(add, remove, check)
    catch e
        e == exittag || rethrow(e)
    end
    return addset, removeset
end

graph_transform() do add, remove, check
   # code block
end
9

Looks about right. Note that you forgot hygiene on a few symbols though (NonlocalTransfer, Set, push!, !, throw, !=, rethrow) which should all be GlobalRef(@__MODULE__, :Set), etc.

Alternatively, add esc() around bodyblock, add, remove, and check and you shouldn’t need to add mangling to anything else.

10

Thanks Jameson.

11

Thanks. Very elegant.

It’s been decades since I’ve read any of Steele’s “Lambda the Ultimate …” papers. It may be time to reread them and reset my head.

2 Likes