Using metaprogramming to combine multiple functions and construct a composite temporary function

General Usage
1

I can identify the input and output variable names of the functions. Using this information, I want to combine these functions to construct a composite function. I hope to obtain the result with the specified variable names without inputting intermediate data. The code is as follows:

# function and its input and output
func_infos = Dict(
    :f1 => ((:x, :y), :z1),
    :f2 => ((:z1,), :z2),
    :f3 => ((:z2,), :z3),
    :f4 => ((:z1, :z3), :z4),
    :f5 => ((:z1,), :z5)
)

# functions
f1(x, y) = x + y + 1
f2(z1) = z1 * 2
f3(z2) = z2 - 3
f4(z1, z3) = (z1 + z3) / 4
f5(z1) = z1 ^ 2

I can identify the input and output variable names of the functions. Using this information, I want to combine these functions to construct a composite function f(x,y) that can obtain the result for z4 without inputting intermediate data.

2

Is func_infos some data you’re given and must work with, or is it just your version of piping syntax? The base |> is only useful for single inputs and outputs, but Pipe.jl uses a macro to make multiple inputs and outputs work. Not sure if inputs could jump to multiple steps like :z1 to :f2 and :f4, though. func_infos could also just be rewritten as temporary variables and function calls, so am I correct in assuming func_infos is intended to be mutated?

3

Pipe.jl seems feasible; I may need to replace each function individually, But I need to test whether it can replace all variables from bottom to top.

4

Indeed, this is feasible, as shown in the following code:

combine_func = (x, y) -> @pipe f1(x, y) |> (x, y, f2(_)) |> (f1(_[1], _[2]), f3(_[3])) |> f4(_[1], _[2])

But this is just an example function. When encountering other similar functions, I can’t manually combine each one. I’m wondering if there’s a general method to combine the above functions.

5

It’s possible to compose functions without resorting to metaprogramming. Perhaps try with CallableExpressions.jl, some other packages are available, too.

6

With these definitions I first refactor the info Dict, and create a makefun function.

# function and its input and output
func_infos = Dict(
    :f1 => ((:x, :y), :z1),
    :f2 => ((:z1,), :z2),
    :f3 => ((:z2,), :z3),
    :f4 => ((:z1, :z3), :z4),
    :f5 => ((:z1,), :z5)
)

# functions
f1(x, y) = x + y + 1
f2(z1) = z1 * 2
f3(z2) = z2 - 3
f4(z1, z3) = (z1 + z3) / 4
f5(z1) = z1 ^ 2

function invertinfo(info)
    result = Dict()
    for (f, (x, y)) in pairs(info)
        result[y] = (f, x)
    end
    return result
end
iinfo = invertinfo(func_infos)


function funsub(y, iinfo)
    y ∈ (:x,:y) && return y
    f, x = iinfo[y]
    xx = funsub.(x, Ref(iinfo))
    quote
        $f($(xx...))
    end
end

function makefun(sym, iinfo)
    @eval (x,y) -> $(funsub(sym, iinfo))
end

It’s used as follows:

f = makefun(:z4, iinfo)
# f is now a function of 2 variables evaluating to the `z4` expression.
f(2, 3)

Depending on what this is used for, it might be wiser to combine expressions, not functions.

1 Like
7

What is the context here that is leading you towards metaprogramming (which is often a bad idea)? Why not just write something like

function f(x, y)
    z1 = f1(x, y)
    z2 = f2(z1)
    z3 = f3(z2)
    z4 = f4(z1, z3)
    z5 = f5(z1)
    return (; z1, z2, z3, z4, z5)
end

if that is the function composition you want? That is, why are you defining a custom function-composition representation using a Dict of symbols rather than using the Julia language itself?

1 Like
8

If I’m reading the docs correctly, CallableExpressions doesn’t introduce more named types like anonymous functions do, right? Not sure if expression_into_type_domain does that. My concern here is that named types persist so a program that evaluates, uses, and discards an arbitrary number of anonymous functions is a big memory leak. I wasn’t even sure of that but I started another thread to check.

9

Sounds like you might be better off with something like DynamicExpressions.jl.

1 Like
10

Yeah, there’s no metaprogramming.

1 Like
11

Thank you all for your valuable suggestions. Here is my final solution:

using RuntimeGeneratedFunctions
RuntimeGeneratedFunctions.init(@__MODULE__)

func1 = (i, p) -> (i[1] + i[2] * p[1], i[1] * p[2])
func1_input_names = [:x1, :x2, :x3]
func1_output_names = [:y1, :y4]
func2 = (i, p) -> i[1] + i[2] * p[1] + i[3] * p[2]
func2_input_names = [:x1, :y1, :x3]
func2_output_names = :y2
func3 = (i, p) -> i[1] + i[2] * p[2]
func3_input_names = [:y1, :y2]
func3_output_names = :y3

exprs = [
    :(($(func1_output_names...),) = $(func1)([$(func1_input_names...)], p)),
    :($(func2_output_names) = $(func2)([$(func2_input_names...)], p)),
    :($(func3_output_names) = $(func3)([$(func3_input_names...)], p))
]

f = @RuntimeGeneratedFunction(
    :((i, p) -> begin
        ($(func1_input_names...),) = i
        $(exprs...)
        return $(func3_output_names)
    end)
)

f([2, 3, 4], [2, 3])

This might differ slightly from the initial approach. Here, I defined various intermediate variables within the function. By defining the outputs of multiple functions, I can use the intermediate outputs as inputs for other functions. This way, my problem is solved. However, this approach relies on the order of function calculations, so it may be worth considering constructing a computational graph to determine the sequence.