Automatically generating methods with a macro

General Usage
1

I was recently implementing a bunch of functions which all had multiple methods with identical structure. The methods differed only in their arguments and some preprocessing steps. EDIT (borrowed from a later post): The general form of these functions can be described like this:

  • We have n functions f_1, f_2, …, f_n.
  • For each function, we have m methods.
  • Each method has the form f_j(x_k,y_j) = (expr_k ; expr_j), where x_k are arguments that depend only on the method and y_j are arguments that depend only on the function, and expr_k is a code block that depends only on the method and expr_j is a code block that depends only on the function.

I would like to be able to automatically generate all the methods for all the functions, while only specifying x_k, y_j, expr_k, and expr_j once. I have a solution for this problem, but I suspect there exist much better solutions.

Here’s a toy example to illustrate the problem, with n=2 and m=2:

struct Foo
    data::Vector{Float64}
    xMax::Real
    n::Int
end

function linearFoo(xMax::Real,n::Int,a::Real,b::Real)
    x = (1:n) .* xMax                                 # Boilerplate
    data = a .+ b .* x                                # This is the important line
    return Foo(data, xMax, n)
end

function linearFoo(f::Foo,a::Real,b::Real)
    xMax = f.xMax                                     # Boilerplate
    n = f.n                                           # Boilerplate
    x = (1:n) .* xMax                                 # Boilerplate
    data = a .+ b .* x                                # This is the important line
    return Foo(data, xMax, n)
end

function quadraticFoo(xMax::Real,n::Int,a::Real,b::Real,c::Real)
    x = (1:n) .* xMax
    data = a .+ b .* x .+ c .* x.^2
    return Foo(data, xMax, n)
end

function quadraticFoo(f::Foo,a::Real,b::Real,c::Real)
    xMax = f.xMax
    n = f.n
    x = (1:n) .* xMax
    data = a .+ b .* x .+ c .* x.^2
    return Foo(data, xMax, n)
end

I’m a complete novice at metaprogramming, but this seemed like something I could automate with a macro. I managed to do so, but I’m not sure that my solution was remotely optimal. I’m interested to know if anyone else has dealt with similar issues and has an elegant solution. Here’s what I came up with:

# Template for the first method
methodPattern1 = quote
    function makeFoo(xMax::Real,n::Int)
         x = (1:n) .* xMax
    end
end

# Template for the second method
methodPattern2 = quote
    function makeFoo(f::Foo)
        xMax = f.xMax
        n = f.n
        x = (1:n) .* xMax
    end
end

# Helpers to extract template data
patternRef_args(x::Expr)         = x.args[2].args[1].args     # push! arguments here
patternRef_body(x::Expr)         = x.args[2].args[2].args     # push! function body here

# Helpers to extract new function data
funcRef_name(x::Expr) = x.args[1].args[1]
funcRef_args(x::Expr) = x.args[1].args[2:end]
funcRef_body(x::Expr) = x.args[2]

# Update the pattern with the new function data
function updateFuncExpr!(pattern,func)
    patternRef_args(pattern)[1] = funcRef_name(func)
    push!(patternRef_args(pattern),funcRef_args(func)...)
    push!(patternRef_body(pattern),funcRef_body(func))
end

# Put it all together
macro addTemplateMethods(funcExpr)
    patt1 = copy(methodPattern1)
    updateFuncExpr!(patt1,funcExpr)
    
    patt2 = copy(methodPattern2)
    updateFuncExpr!(patt2,funcExpr)

    return quote 
        $(esc(patt1))
        $(esc(patt2))
    end
end

Example usage:

@addTemplateMethods function cubicFoo(a::Real,b::Real,c::Real,d::Real)
 data = a .+ b .* x .+ c .* x.^2 .+ d .* x.^3
 return Foo(data, xMax, n)
end

cubicFoo(10,20,1,2,3,4)

The real thing was a bit more complicated, as I handled parametric types and keyword args, but it had the same flavor. It nicely simplifies the process of adding both new functions and new methods (e.g. if I wanted a method that took a Foo argument, but allowed you to overwrite the xMax parameter). However, the metaprogramming code that extracts parts of the Expr tree is very opaque and a bit fragile, and I wonder if there’s a better way to accomplish this.

Any suggestions for how to improve this?

2

Macros are really powerful and fun! I think this example is better solved with multiple dispatch. Is this correct? If so, adding a new case is just implementing foo(args...) with a new number of parameters.

julia> struct Foo{T <: AbstractVector{<:Real}, S <: Real}
           data::T
           xMax::S
           n::Int
       end

julia> Foo(f::Foo, args...) = Foo(f.xMax, f.n, args...);

julia> Foo(xMax::Real, n::Int, args...) = Foo(foo((1:n) * xMax, args...), xMax, n);

julia> foo(x, a, b) = @. a + b * x;

julia> foo(x, a, b, c) = @. a + b * x + c * x ^ 2;

julia> x = Foo(2, 3, 4, 5)
Foo{StepRangeLen{Int64, Int64, Int64, Int64}, Int64}(14:10:34, 2, 3)

julia> y = Foo(x, 6, 7, 8)
Foo{Vector{Int64}, Int64}([52, 162, 336], 2, 3)
1 Like
3

Also, @evalpoly might solve the general case (admittedly I don’t roam this territory often, so someone else please confirm) ah nevermind, it doesn’t handle vectors :smiley:

4

Not quite. I think your solution requires that the different methods have unique signatures, which is not something that holds in general (although coincidentally it did in my toy example). E.g. I might want a function like

function gaussianFoo(f::Foo,w::Real,h::Real)
    xMax = f.xMax  
    n = f.n  
    x = (1:n) .* xMax
    data = a .* exp.(-x.^2 ./ b^2) 
    return Foo(data, xMax, n)
end

I guess the general problem I want to solve looks like this, ignoring complexities like parametric types and keyword args:

  • We have n functions f_1, f_2, …, f_n.
  • For each function, we have m methods.
  • Each method has the form f_j(x_k,y_j) = (expr_k ; expr_j), where x_k are arguments that depend only on the method and y_j are arguments that depend only on the function, and expr_k is a code block that depends only on the method and expr_j is a code block that depends only on the function.
  • I would like to be able to automatically generate all the methods for all the functions, while only specifying x_k, y_j, expr_k, and expr_j once.

Another simple example to keep in mind is a function which can accept either a vector or a tuple as an argument. The methods in either case are almost identical, but you might want to do some processing in one or the other case, e.g. to make sure that the vector has the appropriate length or to do type conversions.

5

How about parameterizing the data function? That way, the boilerplate is still abstract away and you can select which one to call.

EDIT: this still leaves methods for polynomial and guassian to be generated, but it removes a lot of external complexity

julia> Foo(f::Function, xMax::Real, n::Int, args...) = Foo(f((1:n) * xMax, args...), xMax, n);

julia> Foo(gaussian, 2, 3, 4, 5)
6 
julia> @generated function polynomial(x, args...)
           # return `Expr` based on number of parameters
       end
1 Like