@defined "not working" inside macro

General Usage Performance
1

Predictably, when I do metaprogramming, I end up confused.

I create functions that take user-defined function as an input - very flexible. But when the user writes a script to use my function, each time they parse the script, their function is parsed afresh, and so becomes, in the compiler’s eye, a new function, of a new type, even if unchanged by the user. This in turns causes my functions to be recompiled every time the user executes the script.

This motivates my little project: a macro that prevents reparsing of unmodified functions. The user gives a tag (a variable in the user’s scope) to which the code of the function will be assigned, allowing to detect any change in the code

macro once(tag,ex)
    ex  = esc(ex)
    tag = esc(tag)
    return quote
        if ~@isdefined($tag) || $tag≠$ex
            $tag = $ex
            $ex    
        end 
    end
end

I test it with

@once tag_in_callers_scope g(x) = 3

The macro does not compile, I get the error

ERROR: LoadError: MethodError: no method matching var"@isdefined"(::LineNumberNode, ::Module, ::Expr)

Closest candidates are:
  var"@isdefined"(::LineNumberNode, ::Module, ::Symbol)

If I delete ~@isdefined $tag || the macro does generate code (that is not worth executing).

The error message could suggest that while interpolating an escaped Expr containing a single Symbol works as intended in general, in the context of interpolating arguments to a macro, this works differently.

On the other hand, even if I do not escape tag (it’s then a Symbol, I checked with typeof), I get the same error (code not worth executing).

And then, if instead of taking tag as an input to @once, I generate it inside the macro with tag = gensym("tag"), the macro compiles (code not worth executing).

I am at the stage where the only question I can formulate is “what on earth…?”.

:thinking:

2

I think you probably want this.

       macro once(tag,ex)
           return esc(quote
               if ~@isdefined($tag) || $tag≠$ex
                   $tag = $ex
                   $ex    
               end 
           end)
       end

The error message told you that you tried to pass an expression to the macro@isdefined. That’s correct. You are passing Expr(:escape, tag) to it. What you want to do escape the entire final expression.

3

Hi @mkitti

The code generated by your version of the macro (having cleaned line numbers…) is

if ~($(Expr(:isdefined, :tag))) || tag ≠ (g(x) = 4)
      tag = (g(x) = 4)
      g(x) = 4
  end

where it should be

if ~@isdefined(tag) || tag ≠ (g(x) = 4)
      tag = (g(x) = 4)
      g(x) = 4
  end

So escaping the whole expression (instead of symbols) does not seem to work.

4

That seems to be what it is supposed to expand to anyway:

julia> @macroexpand if ~@isdefined(tag) || tag ≠ (g(x) = 4)
                      tag = (g(x) = 4)
                      g(x) = 4
                    end
:(if ~($(Expr(:isdefined, :tag))) || tag ≠ (g(x) = begin
                      #= REPL[10]:1 =#
                      4
                  end)
      #= REPL[10]:2 =#
      tag = (g(x) = begin
                  #= REPL[10]:2 =#
                  4
              end)
      #= REPL[10]:3 =#
      g(x) = begin
              #= REPL[10]:3 =#
              4
          end
  end)

A function that returns the escaped quote doesn’t expand the @isdefined call, that only occurs when the surrounding quote is evaluated, like after the @once method returns.

5

:rofl:
I just examined the output (@macroexpand) and didn’t like it. I didn’t even check what the macro did, and indeed it works: @mkitti solved the problem! Thank you, guys!

That said, it’s all a bit magic to me. What’s wrong with my original code, and what, exactly, is the effect of esc on a whole expression - apparently it’s not recursively going down the tree and then escaping individual symbols.

Anyway, for anyone interested, the following code not only compiles, but even does as expected:

  • The QuoteNode allows source codes to be compared
  • The postwalk operations (MacroTools.jl) clean out things like line numbers which, while useful in error callstacks would make the exact same source code, with a new empty line on top, be a “new” function.
using MacroTools

macro once(tag,ex)
    ex  = postwalk(rmlines,ex)
    ex  = postwalk(unblock,ex)
    qex = QuoteNode(ex)
    return esc(quote
        if  ~@isdefined($tag) || $tag≠$qex
            $tag = $qex
            $ex    
        end 
    end)
end
6

esc just informs the macro expansion some variables should come from the call’s scope, not the definition’s scope. The issue appears to be that @isdefined doesn’t handle escaped expressions as inputs, only symbols, so there’s only one working order. Here’s a shorter reproduction of your error:

julia> @macroexpand esc(@isdefined(tag))
:(esc($(Expr(:isdefined, :tag))))

julia> @macroexpand @isdefined($(esc(:tag)))
ERROR: MethodError: no method matching var"@isdefined"(::LineNumberNode, ::Module, ::Expr)
The function `@isdefined` exists, but no method is defined for this combination of argument types.

Closest candidates are:
  var"@isdefined"(::LineNumberNode, ::Module, ::Symbol)
...

I’m still not clear on how this macro would help this scenario. The macro doesn’t seem to stop redefinitions at all because the condition $tag≠$ex still evaluates a method redefinition ex prior to the comparison of the tag and the function (not method) returned by the redefinition:

julia> foo() = 1
foo (generic function with 1 method)

julia> @once footag foo()=2 # returns function, so conditional statement ran
foo (generic function with 1 method)

julia> foo()
2

julia> @once footag foo()=3 # returns nothing, so conditional statement did not run

julia> foo() # still redefined by condition
3

Re-running a possibly edited script into the same module also doesn’t typically generate a new function and type, though the method redefinitions do trigger recompilation of the methods and their callers. But the approach is to instead evaluate only the edited methods, possibly with the help of Revise.jl. What you intend for your users must be very different, and if that’s described more precisely in a MWE, there may be something to do about it.

7

Hi @Benny

With the help from you both, the macro compiled, and thanks. But my code was still erroneous. I corrected it and posted above.

OK…! If that is the case, @once does precisely nothing and I misunderstood the reasons for my compilation times.

Testing:

I define

@generated function foo(f)
    @show :generating
    return 0
end

to be able to spy on recompilation.

Then I repeatedly run the script

h(x) = 1
foo(h)

but :generating is only shown the first time. You are absolutely right and @once is unnecessary!

Good to know? Ignorance is bliss? :smiley:

8

That’s only because you don’t do anything with the input variable f. If you called it, you’d observe recompilation upon redefinitions, even if the method body didn’t change:

julia> @generated function foo(f)
           @show :generating
           return :(f())
       end
foo (generic function with 1 method)

julia> h()=0; foo(h)
:generating = :generating
0

julia> h()=0; foo(h)
:generating = :generating
0

julia> h()=2; foo(h)
:generating = :generating
2

The same thing happens to normal functions, and you can spot any hint of recompilation with @time @eval foo(h) as documented. @generated functions are especially tricky, don’t resort to them for reflection.

It’s very likely that redefinitions of the method, even with the same method body, forced recompilation of callers as demonstrated. Again, the typical approach is to avoid doing that, but it’s still not clear what you plan for users.

9

Ahaha!!! Then it seems @once has its use after all. Using your generated function, the script

@once mytag h3() = 2
foo(h3)

shows :generating only the first time it is executed, while

h3() = 2
foo(h3)

shows it every time. :smiley:

I note your warning on @generated for reflection though.

Let me try to explain the usecase: foo is (well: stands for) part of the functionality delivered by my package (I am developing Muscade.jl), and some of the package-user’s input is given in the form of functions h3 they define.

Muscade is an inverse-FEM software, and the “input file” describing their model will be a Julia script with multiple calls to various constructors to define the model, nodes, elements etc., before feeding the model into a solver. I think it’s pretty classic Julia. For some of the element constructors foo, some of the input is a used-defined function h3. h3 will typically be very light weight, but recompiling some of the methods of the object constructed by foo can take a handful of seconds.

My instruction to the user is then: anotate the functions you define in the script with @once.