Macro: is it possible to receive an evaluated argument?

I’d like to manipulate function arguments easily and I expect that macro can do this.
However, I probably misunderstood the usage of macro.

For example, the following is a brief (probably not runnable) code describing my intention.

  • MyModule.jl
module MyModule

abstract type AbstractMyStruct end
macro  mymacro(ex, mystruct::AbstractMyStruct)
   ...
end

end
  • test.jl
include("MyModule.jl")
using .MyModule

struct MyStruct <: AbstractMyStruct
end
f(a, b) = a * b
mystruct = MyStruct()
value(mystruct::MyStruct) = 2
@mymacro(f(a, b), mystruct)  # == f(a, b*value(mystruct))

In a test I personally did, it is hard for mymacro to receive (evaluated) mystruct in MyModule.jl.

A macro always only receives the syntax tree in the form of Expr objects or literals like Int, String etc that have a direct code representation. So you can’t feed it a custom struct, it only sees the symbol that is your struct’s variable name. Imagine the macro just transforming the code you have written, and afterwards that transformed code is run. Only in that running phase does your struct exist.

What I find really helpful for learning is to write a helper macro

macro test(expr)
    dump(expr)
end

Run @test some code and check out what you actually receive inside the macro. That structure is what you can work with.

1 Like

Then is there no way to achieve what I want using macro? :frowning:
It would be great if it’s possible…

You mean this line should hold?

That’s certainly possible, but again don’t think about it in terms of your custom struct etc. You just take two expressions, one is the Expr f(a, b) and one is the symbol mystruct. The you check if the first Expr is a call expression with two arguments that are both symbols. If it is, replace the second argument with a call expression that is a multiplication of the second argument of f and the second argument of the macro.

1 Like

Fundamentally, what is a macro? A macro transforms syntax into syntax, there’s no values involved here (in some sense, the values that will eventually be computed don’t exist during macro expansion at all - there’s only parsed syntax, none of the modified code has run yet). Let’s take a look at what we have (f(a,b) and :mystruct) and what we want (f(a, b*value(mystruct))). What do these two look like once they’re parsed, i.e. what does the julia compiler see? We can find out via Meta.@dump:

julia> Meta.@dump (f(a,b), mystruct)    
Expr                                    
  head: Symbol tuple                    
  args: Array{Any}((2,))                
    1: Expr                             
      head: Symbol call                 
      args: Array{Any}((3,))            
        1: Symbol f                     
        2: Symbol a                     
        3: Symbol b                     
    2: Symbol mystruct                  

This may look scary at first, so let’s break it down. This expression tells us what @dump saw is a tuple of 2 elements (hence head is :tuple and args is Array{Any}((2,))). The two arguments are another expression (which is the way function calls are represented) and a Symbol (in this case, :mystruct).

Now for the second expression:

julia> Meta.@dump f(a,b*value(mystruct))
Expr                                    
  head: Symbol call                     
  args: Array{Any}((3,))                
    1: Symbol f                         
    2: Symbol a                         
    3: Expr                             
      head: Symbol call                 
      args: Array{Any}((3,))            
        1: Symbol *                     
        2: Symbol b                     
        3: Expr                         
          head: Symbol call             
          args: Array{Any}((2,))        
            1: Symbol value             
            2: Symbol mystruct          

This is what we want to get out of the macro. Notice how this expression immediately starts with :call! It’s not wrapped in a :tuple expression anymore. We can also see that the third argument to the call (which used to be just :b above) is now another expression calling *, which itself has another :call embedded in it, this time to the function value. The whole inner call with :value and :mystruct corresponds to value(mystruct). Now, how do we get from the first form to the second form? We simply build the expression in our macro and return it:

macro myMacro(ex, mystruct)
  #do error handling here, e.g. if `ex` is not a function call!
  local tmp = ex.args[3] # get the third argument, which is `b`
  local valueExpr = :(value($mystruct))
  local multiplyExpr = Expr(:call, :*, tmp, valueExpr)
  ex.args[3] = multiplyExpr
  return ex
end

Using this, we get:

julia> struct MyStruct                                                
       end                                                            
                                                                      
julia> f(a, b) = a * b                                                
f (generic function with 1 method)                                    
                                                                      
julia> mystruct = MyStruct()                                          
MyStruct()                                                            
                                                                      
julia> value(mystruct::MyStruct) = 2                                  
value (generic function with 1 method)                                
                                                                      
julia> macro myMacro(ex, mystruct)                                    
         #do error handling here, e.g. if `ex` is not a function call!
         local tmp = ex.args[3] # get the third argument, which is `b`
         local valueExpr = :(value($mystruct))                        
         local multiplyExpr = Expr(:call, :*, tmp, valueExpr)         
         ex.args[3] = multiplyExpr                                    
         return ex                                                    
       end                                                            
@myMacro (macro with 1 method)                                        
                                                                      
julia> a = 3 # define our variables
3                                                                     
                                                                      
julia> b = 4 # define our variables
4                                                                     
                                                                      
julia> @macroexpand @myMacro(f(a,b), mystruct)                        
:(Main.f(Main.a, Main.b * Main.value(Main.mystruct)))  # sanity check, this looks like what we care about
                                                                      
julia> @myMacro(f(a,b), mystruct)                                     
24 # 3*4*2 = 24, seems to have worked!

I hope this helps!

On a side note, the way this macro is written is very crude and should use some sanity checking and escaping logic, to make sure the correct variables are referenced in all contexts :slight_smile: This is called “Macro Hygiene”, the documentation as a great chapter about it here.

If we don’t practice hygiene, the macro won’t work correctly in other scopes:

julia> function x()                                     
         a = 0                                          
         b = 0                                          
         mystruct = MyStruct()                          
         @myMacro(f(a,b), mystruct)                     
       end                                              
x (generic function with 1 method)                      
                                                        
julia> x()                                              
24 # we defined `a` and `b` to be `0` in `x()`, so we would expect to get `0` out as well! Why don't we?
                                                        
# Let's find out why by checking with `@macroexpand`
julia> @macroexpand function x()                        
         a = 0                                          
         b = 0                                          
         mystruct = MyStruct()                          
         @myMacro(f(a,b), mystruct)                     
       end                                              
:(function x()                                          
      #= REPL[25]:1 =#                                  
      #= REPL[25]:2 =#                                  
      a = 0                                             
      #= REPL[25]:3 =#                                  
      b = 0                                             
      #= REPL[25]:4 =#                                  
      mystruct = MyStruct()                             
      #= REPL[25]:5 =#                                  
      Main.f(Main.a, Main.b * Main.value(Main.mystruct)) # Here's the culprit! Everything is referencing `Main`, which it clearly shouldn't here...
  end)                                                  
3 Likes

Nice, I didn’t know Meta.@dump so I don’t have to write my little helper anymore

It’s a really nice explanation. Thank you :slight_smile:

I wonder if it’s possible to provide a macro in a separated module.
For example, I wrote your example in two scripts, one is a module and the other is a test script as

  • MyModule.jl
module MyModule
    function value end
    macro myMacro(ex, mystruct)
        local tmp = ex.args[3]
        local valueExpr = :(value($mystruct))
        local multiplyExpr = Expr(:call, :*, tmp, valueExpr)
        ex.args[3] = multiplyExpr
    return ex
end
end
  • test.jl
using MyModule

struct MyStruct
end
mystruct = MyStruct()

MyModule.value(mystruct::MyStruct) = 2

f(a, b) = a*b
@MyModule.myMacro(f(1, 2), mystruct)
  • error message
ERROR: LoadError: UndefVarError: mystruct not defined

For me, the error message is actually quite obvious. Is it hard to deal with this situation?

It absolutely is possible, but that’s the point where macro hygiene comes into play :slight_smile: From a high level, the problem you’re encountering is the same as the issue with using the macro in a function, it just manifests differently. For more information on how to practice macro hygiene, see the link to the docs:

https://docs.julialang.org/en/v1/manual/metaprogramming/#Hygiene

1 Like

In this case, it helps to look at the result of your macro transformation with @macroexpand (I just pasted the code into one file and ran it in Main):

julia> @macroexpand @MyModule.myMacro(f(1, 2), mystruct)
:(Main.MyModule.f(1, 2 * Main.MyModule.value(Main.MyModule.mystruct)))

So you see that all symbols are qualified with the module where the macro is defined. That’s the default hygienic behavior. It’s useful because often in macros, you want to reference variables from the macro module, and you don’t want random variables added to whatever space where the macro is run (as a library author you have no idea what other variables people have in their code, so anything you add could collide). And of course MyModule.mystruct doesn’t exist. So you need to escape that symbol, and also f because it also doesn’t exist in MyModule but in Main. Escaping means that a symbol will end up in the expression “as-is”.

If you add this macro in your module:

 macro myMacroEscaped(ex, mystruct)
        local tmp = ex.args[3]
        local valueExpr = :(MyModule.value($mystruct))
        local multiplyExpr = Expr(:call, :*, tmp, valueExpr)
        ex.args[3] = multiplyExpr
    return esc(ex)
end

You can see that I escaped the whole expression at the end, but I also qualified value as MyModule.value because otherwise you get the inverse problem that value is not defined in Main.
Then you get:

julia> @macroexpand @MyModule.myMacroEscaped(f(1, 2), mystruct)
:(f(1, 2 * MyModule.value(mystruct)))

Basically, with escaping, you usually want to escape variables that come from the macro-calling code, because those are added by the user, while you don’t want to escape any of your own variables relating to stuff from the macro module.

1 Like