Help with macro calling another macro

New to Julia
1

So let’s say I have some code:

if !@isdefined(a)
    a = 1
end
if !@isdefined(b)
     b = 2
end

I’d like to refactor it into something like:

macro defifndef(x, y)
    if !@isdefined($x)
        $x = $y
    end
end

@defifndef(a, 1)
@defifndef(b, 2)

However, the above code doesn’t work and I’m getting lost trying to figure out what combination of quotes and evals would give me the desired result. I tried some variations based on Call a macro within a macro and Calling a macro from within a macro, revisited but I can’t figure it out.

Any suggestions would be much appreciated, and thanks in advance!

2

First thing you probably want to do is to quote the returned expression so it is not evaluated in the macro.

macro defifndef(x, y)
    quote
        if !@isdefined($x)
            $x = $y
        end
    end
end

But testing this we get

julia> @defifndef a 3
3

julia> a
ERROR: UndefVarError: a not defined

so let’s have a look at what is going on.

julia> @macroexpand @defifndef a 3
quote
    #= REPL[1]:3 =#
    if !($(Expr(:isdefined, Symbol("#15#a"))))
        #= REPL[1]:4 =#
        var"#15#a" = 3
    end
end

Here we see that it is assigning 3 to the variable var"#15#a" which is not what we wanted. We need to escape this to tell julia that this is not a variable we want it to be smart and hygenic about.

macro defifndef(x, y)
    quote
        if !@isdefined($(esc(x)))
            $(esc(x)) = $y
        end
    end
end

Sadly this does not work either, and now I realised I’m late for something else so have to leave. But maybe I gave you some tips at least. Macros can be tricky to get correct…

3

Probably you want esc($x) not the other way.

4

My tip for writing macros, always structure them like this:

macro mymacro(x)
    esc(mymacro_helper(x))
end

function mymacro_helper(x)
    # Act on the expression x
end

And then in mymacro_helper, you return the expression you want, constructed through quote ... end.

It’s nice to separate the stuff that acts on the expression from the macro hygiene issues.

1 Like
5

You lose macro hygiene if you escape everything, as opposed to just escaping x.

1 Like
6

Can you please give an example? I’ve been using that pattern a lot.

7

You should only escape where you know you have some symbols from the user that need to make it back out unchanged. Everything that you reference in your macro should be unescaped, otherwise all your symbols need to be available in the calling scope. For example, you use transform and escape, then transform needs to be in scope. Maybe users didn’t do using DataFrames though. Same problem with DataFrames.transform, then DataFrames needs to be in scope. If you don’t escape, you’ll always have access to the symbols that are available in your macro module, the transform example will be DataFrameMacros.DataFrames.transform for example.

2 Likes
8

Consider these 3 variants of a timing macro:

This one doesn’t escape anything, causing the user-defined variable to be left undefined:

julia> macro my_time(expr)
           quote
               start = time()
               retval = $expr
               println("Elapsed time: $(time() - start)s.")
           end
       end
@my_time (macro with 1 method)

julia> @my_time begin
           sleep(1)
           a = 42
       end
Elapsed time: 1.0227389335632324s.

julia> start   # This is good
ERROR: UndefVarError: start not defined

julia> a  # But not this
ERROR: UndefVarError: a not defined

If you escape everything, user-defined names are correctly accounted for, but you lose hygiene: in this case for example the macro leaks the variables it uses internally and could cause name collisions with user-defined variables:

julia> macro my_time(expr)
           quote
               start = time()
               retval = $expr
               println("Elapsed time: $(time() - start)s.")
           end |> esc
       end
@my_time (macro with 1 method)

julia> @my_time begin
           sleep(1)
           a = 42
       end
Elapsed time: 1.023123025894165s.

julia> a # This is fixed
42

julia> start  # This should not be defined; it comes from the macro itself
1.637597519544262e9

Now this last version escapes only the user-provided code, making it more hygienic:

julia> macro my_time(expr)
           quote
               start = time()
               retval = $(esc(expr))
               println("Elapsed time: $(time() - start)s.")
           end
       end
@my_time (macro with 1 method)

julia> @my_time begin
           sleep(1)
           a = 42
       end
Elapsed time: 1.0229160785675049s.

julia> a  # OK
42

julia> start   # OK
ERROR: UndefVarError: start not defined
5 Likes
9

Okay, thanks for this. I will look through all my code and try and fix things.

I think that the point made by @jules is more relevant, since careful use of gensym() can get rid of the problem described by @ffevotte , and I’ve been good about using gensym() properly to avoid leaks.

10

It’s not just leaks outwards, of symbols defined in the macro existing outside, as in the @my_time example. But also inwards, of symbols re-defined elsewhere affecting what the macro does:

julia> let println = throw
              @my_time begin  # version of @my_time which escapes everything
                  sleep(1)
                  a = 42
              end
       end
ERROR: "Elapsed time: 1.0061688423156738s."

julia> macro defifndef1(x, y)
           :(if !(@isdefined($x))
               $x = sqrt($y)
           end) |> esc  # escapes everything, including sqrt
       end;

julia> @macroexpand1 @defifndef1(a, 2)
:(if !(#= REPL[8]:2 =# @isdefined(a))
      #= REPL[8]:3 =#
      a = sqrt(2)
  end)

julia> @defifndef1 a 2

julia> let sqrt = cbrt
         @defifndef1 b 2  # wrong answer! 
       end
1.2599210498948732

julia> macro defifndef2(x, y)
           :(if !(@isdefined($x))  # weirdly, not esc(x) here
               $(esc(x)) = sqrt($(esc(y)))  # now uses sqrt from definition's scope
           end)
       end;

julia> @macroexpand1 @defifndef2(a, 2)
:(if !(#= REPL[23]:2 =# @isdefined(a))
      #= REPL[23]:3 =#
      a = Main.sqrt(2)
  end)

julia> let sqrt = cbrt
         @defifndef2 c 2  # now this works
       end
1.4142135623730951

julia> macro defifndef3(x, y)
           ex = Expr(:macrocall, Symbol("@isdefined"), nothing, x) # not esc(x)
           :(if !($ex)
               $(esc(x)) = sqrt($(esc(y)))
           end)
       end;

julia> @macroexpand1 @defifndef3(a, 2)
:(if !(@isdefined(a))
      #= REPL[12]:4 =#
      a = Main.sqrt(2)
  end)

julia> let sqrt = cbrt
       @defifndef3 c 2
       end
1.4142135623730951

These examples are artificial but someone using (say) sum as the name of a local variable is something that will happen, and will break your macro if it intended to call Base.sum. You could fix @defifndef1 by writing $sqrt to use the one from the defining scope (and probably $! too). That, in addition to gensym for symbols defined within the macro, is generally safe.

But I don’t think you can write $@isdefined. Perhaps you don’t have to, e.g. let var"@isdefined" = var"@show" changes nothing. And, surprisingly, you do not need to escape its argument.

3 Likes
11

That does not seem to work either, gives the same error about @isdefined wanting a symbol not an expression.

My thought was that I want to evaluate the esc so it is not left in the returned expression since @isdefined wants a symbol, so I thought I wanted to have the esc inside the interpolation would make it not return with the expression. Though I have realised this is probably not how it works, and that the esc actually needs to be in the returned expression and is later stripped by the macro expander (unclear about this, but was my conclusion).

It seems that $(esc(x)) will generate this

quote
    #= REPL[16]:3 =#
    if !(#= REPL[16]:3 =# @isdefined($(Expr(:escape, :a))))
        #= REPL[16]:4 =#
        $(Expr(:escape, :a)) = 1
    end
end

while esc($x) will generate this

quote
    #= REPL[20]:3 =#
    if !(#= REPL[20]:3 =# @isdefined(esc(a)))
        #= REPL[20]:4 =#
        esc(a) = begin
                #= REPL[20]:4 =#
                1
            end
    end
end

and I’m not sure which is closer to right.

Saw that while writing this @mcabbott had the solution. Apparently it should be esc inside, but for the one in @isdefined there should only be interpolation. I don’t really understand why.

12

Thanks for more detail.

I’ve opened up This issue at DataFramesMeta.jl to keep track of this. It looks like fixes aren’t trivial due to shared nested functions for constructing the output expression. I would appreciate a more expert view.