Macro that generates additional methods of the same function

General Usage
1

Here is a macro that works almost as intended:

macro gen_argfilt(func, tpl)
       return quote
                  function arg_filt(dat::Union{Model_data, Batch_view},nnw::Wgts,
                     hp::Hyper_parameters, bn::Batch_norm_params, hl::Int, fn::typeof($func)); 
                     $tpl
                  end
               end
end

The arguments are an existing function and a tuple built up of references to the arguments to the output arg_filt method. An example is more obvious:

arg_filt = @gen_arg_filt(relu!,(dat.a[hl], dat.z[hl]))
#113#arg_filt (generic function with 1 method)

And you use the output function like this:

relu!(arg_filt(train, nnw, hp, bn, 1,relu!)...)

The effect is to call the function relu! and recast the arguments to match the tuple that was input to the argument. Don’t ask. OK, why? It defers evaluation of the arguments which are giant arrays. To be useful, arg_filt dispatches on the type of the function.

But, here is the problem. The output of the macro generates funny function names. So, instead of getting new methods of the same function (the function that should be called arg_filt) each function has a funny name like you see above: #113#arg_filt. This is something macros do according to the manual: “Local variables are then renamed to be unique (using the gensym function, which generates new symbols)”.

But, I really want to match the original function name if I run the macro multiple times (which is the whole point). I want a bunch of methods of arg_filt that dispatch on the typeof(afunc). I don’t know if the macro considers the function definition to be local, which seems to be what triggers gensym. Apparently, I can wrap with esc() to avoid gensym, but I can’t figure out what to escape.

I also tried using function closures. It worked and was straightforward, but ran more than 6x slower. The closure is wildly slow when capturing large arrays, which is a known problem identified in the Performance Tips section of the manual. I tried the suggestions, but they didn’t improve performance.

The obvious work-around, which is what I do NOW (and it works) is to have a bunch of ordinary function definitions for all of the methods in the source code. Nothing wrong here and it is perfectly obvious but there are now 23 method definitions and this will increase. I also want to make it easy for users to create a matching method for their own functions. Since the only difference between them is the name of the function and the tuple, it seemed easy to use a code generation approach.

Here is one such method definition, which makes what the macro is supposed to do a lot more obvious:

    function argfilt(dat::Union{Model_data, Batch_view}, nnw::Wgts, hp::Hyper_parameters, 
        bn::Batch_norm_params, hl::Int, fn::typeof(sigmoid!))
        (dat.a[hl], dat.z[hl])
    end

Basically, the macro should make it easy to create a lot more such methods.

But, for all of these gnarly reasons, it is not so easy at it seemed. Again, sorry this is complex and long but some topics just are that way. I am sorry this is all so complicated, but then nearly anything pertaining to macros is inherently complicated.

I am just looking for a way for the macro to always output a function (e.g., method) with the same exact name every time,

Thanks in advance,
Lewis

2

This looks like it might solve it:

macro gen_argfilt(func, tpl, fname)
       return quote
           function $(esc(fname))(dat::Union{Model_data, Batch_view},nnw::Wgts,
               hp::Hyper_parameters, bn::Batch_norm_params, hl::Int, fn::typeof($func)) 
               $tpl
           end # function
        end # quote block
end # macro definition
1 Like
3

The section about “macro hygiene” in the manual is probably useful: Metaprogramming · The Julia Language

2 Likes