# Using metaprogramming to combine multiple functions and construct a composite temporary function

**URL:** <https://discourse.julialang.org/t/using-metaprogramming-to-combine-multiple-functions-and-construct-a-composite-temporary-function/116647>\
**Category:** General Usage\
**Tags:** metaprogramming\
**Created:** [July 5, 2024, 8:16am UTC](https://discourse.julialang.org/t/using-metaprogramming-to-combine-multiple-functions-and-construct-a-composite-temporary-function/116647 "2024-07-05T08:16:42Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![chooron](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chooron/32/208300_2.png) [@chooron](https://discourse.julialang.org/u/chooron)\
**Post date:** [July 5, 2024, 8:16am UTC](https://discourse.julialang.org/t/using-metaprogramming-to-combine-multiple-functions-and-construct-a-composite-temporary-function/116647/1 "2024-07-05T08:16:42Z")

</div>

I can identify the input and output variable names of the functions. Using this information, I want to combine these functions to construct a composite function. I hope to obtain the result with the specified variable names without inputting intermediate data. The code is as follows:

```julia
# function and its input and output
func_infos = Dict(
    :f1 => ((:x, :y), :z1),
    :f2 => ((:z1,), :z2),
    :f3 => ((:z2,), :z3),
    :f4 => ((:z1, :z3), :z4),
    :f5 => ((:z1,), :z5)
)

# functions
f1(x, y) = x + y + 1
f2(z1) = z1 * 2
f3(z2) = z2 - 3
f4(z1, z3) = (z1 + z3) / 4
f5(z1) = z1 ^ 2

```

I can identify the input and output variable names of the functions. Using this information, I want to combine these functions to construct a composite function f(x,y) that can obtain the result for z4 without inputting intermediate data.

---

<div class="post-metadata">

**Author:** ![Benny](https://avatars.discourse-cdn.com/v4/letter/b/49beb7/32.png) [@Benny](https://discourse.julialang.org/u/Benny)\
**Post date:** [July 5, 2024, 8:56am UTC](https://discourse.julialang.org/t/using-metaprogramming-to-combine-multiple-functions-and-construct-a-composite-temporary-function/116647/2 "2024-07-05T08:56:25Z")

</div>

Is `func_infos` some data you’re given and must work with, or is it just your version of piping syntax? The base `|>` is only useful for single inputs and outputs, but Pipe.jl uses a macro to make multiple inputs and outputs work. Not sure if inputs could jump to multiple steps like `:z1` to `:f2` and `:f4`, though. `func_infos` could also just be rewritten as temporary variables and function calls, so am I correct in assuming `func_infos` is intended to be mutated?

---

<div class="post-metadata">

**Author:** ![chooron](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chooron/32/208300_2.png) [@chooron](https://discourse.julialang.org/u/chooron)\
**Post date:** [July 5, 2024, 9:09am UTC](https://discourse.julialang.org/t/using-metaprogramming-to-combine-multiple-functions-and-construct-a-composite-temporary-function/116647/3 "2024-07-05T09:09:25Z")

</div>

Pipe.jl seems feasible; I may need to replace each function individually, But I need to test whether it can replace all variables from bottom to top.

---

<div class="post-metadata">

**Author:** ![chooron](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chooron/32/208300_2.png) [@chooron](https://discourse.julialang.org/u/chooron)\
**Post date:** [July 5, 2024, 9:26am UTC](https://discourse.julialang.org/t/using-metaprogramming-to-combine-multiple-functions-and-construct-a-composite-temporary-function/116647/4 "2024-07-05T09:26:30Z")

</div>

Indeed, this is feasible, as shown in the following code:

```julia
combine_func = (x, y) -> @pipe f1(x, y) |> (x, y, f2(_)) |> (f1(_[1], _[2]), f3(_[3])) |> f4(_[1], _[2])

```

But this is just an example function. When encountering other similar functions, I can’t manually combine each one. I’m wondering if there’s a general method to combine the above functions.

---

<div class="post-metadata">

**Author:** ![nsajko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nsajko/32/221187_2.png) [@nsajko](https://discourse.julialang.org/u/nsajko)\
**Post date:** [July 5, 2024, 10:51am UTC](https://discourse.julialang.org/t/using-metaprogramming-to-combine-multiple-functions-and-construct-a-composite-temporary-function/116647/5 "2024-07-05T10:51:41Z")

</div>

It’s possible to compose functions without resorting to metaprogramming. Perhaps try with CallableExpressions.jl, some other packages are available, too.

---

<div class="post-metadata">

**Author:** ![sgaure](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sgaure/32/14779_2.png) [@sgaure](https://discourse.julialang.org/u/sgaure)\
**Post date:** [July 5, 2024, 1:11pm UTC](https://discourse.julialang.org/t/using-metaprogramming-to-combine-multiple-functions-and-construct-a-composite-temporary-function/116647/6 "2024-07-05T13:11:09Z")

</div>

With these definitions I first refactor the info Dict, and create a `makefun` function.

```julia
# function and its input and output
func_infos = Dict(
    :f1 => ((:x, :y), :z1),
    :f2 => ((:z1,), :z2),
    :f3 => ((:z2,), :z3),
    :f4 => ((:z1, :z3), :z4),
    :f5 => ((:z1,), :z5)
)

# functions
f1(x, y) = x + y + 1
f2(z1) = z1 * 2
f3(z2) = z2 - 3
f4(z1, z3) = (z1 + z3) / 4
f5(z1) = z1 ^ 2

function invertinfo(info)
    result = Dict()
    for (f, (x, y)) in pairs(info)
        result[y] = (f, x)
    end
    return result
end
iinfo = invertinfo(func_infos)

function funsub(y, iinfo)
    y ∈ (:x,:y) && return y
    f, x = iinfo[y]
    xx = funsub.(x, Ref(iinfo))
    quote
        $f($(xx...))
    end
end

function makefun(sym, iinfo)
    @eval (x,y) -> $(funsub(sym, iinfo))
end

```

It’s used as follows:

```julia
f = makefun(:z4, iinfo)
# f is now a function of 2 variables evaluating to the `z4` expression.
f(2, 3)

```

Depending on what this is used for, it might be wiser to combine expressions, not functions.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [July 5, 2024, 5:05pm UTC](https://discourse.julialang.org/t/using-metaprogramming-to-combine-multiple-functions-and-construct-a-composite-temporary-function/116647/7 "2024-07-05T17:05:20Z")

</div>

> [@chooron](#):
>
> I can identify the input and output variable names of the functions. Using this information, I want to combine these functions to construct a composite function f(x,y) that can obtain the result for z4 without inputting intermediate data.

What is the context here that is leading you towards metaprogramming (which is [often a bad idea](https://discourse.julialang.org/t/how-to-warn-new-users-away-from-metaprogramming/35022))? Why not just write something like

```julia
function f(x, y)
    z1 = f1(x, y)
    z2 = f2(z1)
    z3 = f3(z2)
    z4 = f4(z1, z3)
    z5 = f5(z1)
    return (; z1, z2, z3, z4, z5)
end

```

if that is the function composition you want? That is, why are you defining a custom function-composition representation using a `Dict` of symbols rather than using the Julia language itself?

---

<div class="post-metadata">

**Author:** ![Benny](https://avatars.discourse-cdn.com/v4/letter/b/49beb7/32.png) [@Benny](https://discourse.julialang.org/u/Benny)\
**Post date:** [July 5, 2024, 6:48pm UTC](https://discourse.julialang.org/t/using-metaprogramming-to-combine-multiple-functions-and-construct-a-composite-temporary-function/116647/8 "2024-07-05T18:48:00Z")

</div>

If I’m reading the docs correctly, CallableExpressions doesn’t introduce more named types like anonymous functions do, right? Not sure if `expression_into_type_domain` does that. My concern here is that named types persist so a program that evaluates, uses, and discards an arbitrary number of anonymous functions is a big memory leak. I wasn’t even sure of that but I started another thread to check.

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [July 5, 2024, 7:20pm UTC](https://discourse.julialang.org/t/using-metaprogramming-to-combine-multiple-functions-and-construct-a-composite-temporary-function/116647/9 "2024-07-05T19:20:07Z")

</div>

> [@Benny](#):
>
> My concern here is that named types persist so a program that evaluates, uses, and discards an arbitrary number of anonymous functions is a big memory leak.

Sounds like you might be better off with something like [DynamicExpressions.jl](https://github.com/SymbolicML/DynamicExpressions.jl).

---

<div class="post-metadata">

**Author:** ![nsajko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nsajko/32/221187_2.png) [@nsajko](https://discourse.julialang.org/u/nsajko)\
**Post date:** [July 5, 2024, 7:37pm UTC](https://discourse.julialang.org/t/using-metaprogramming-to-combine-multiple-functions-and-construct-a-composite-temporary-function/116647/10 "2024-07-05T19:37:09Z")

</div>

> [@Benny](#):
>
> If I’m reading the docs correctly, CallableExpressions doesn’t introduce more named types like anonymous functions do, right?

Yeah, there’s no metaprogramming.

---

<div class="post-metadata">

**Author:** ![chooron](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chooron/32/208300_2.png) [@chooron](https://discourse.julialang.org/u/chooron)\
**Post date:** [July 7, 2024, 2:35am UTC](https://discourse.julialang.org/t/using-metaprogramming-to-combine-multiple-functions-and-construct-a-composite-temporary-function/116647/11 "2024-07-07T02:35:04Z")

</div>

Thank you all for your valuable suggestions. Here is my final solution:

```julia
using RuntimeGeneratedFunctions
RuntimeGeneratedFunctions.init(@ __MODULE__ )

func1 = (i, p) -> (i[1] + i[2] * p[1], i[1] * p[2])
func1_input_names = [:x1, :x2, :x3]
func1_output_names = [:y1, :y4]
func2 = (i, p) -> i[1] + i[2] * p[1] + i[3] * p[2]
func2_input_names = [:x1, :y1, :x3]
func2_output_names = :y2
func3 = (i, p) -> i[1] + i[2] * p[2]
func3_input_names = [:y1, :y2]
func3_output_names = :y3

exprs = [
    :(($(func1_output_names...),) = $(func1)([$(func1_input_names...)], p)),
    :($(func2_output_names) = $(func2)([$(func2_input_names...)], p)),
    :($(func3_output_names) = $(func3)([$(func3_input_names...)], p))
]

f = @RuntimeGeneratedFunction(
    :((i, p) -> begin
        ($(func1_input_names...),) = i
        $(exprs...)
        return $(func3_output_names)
    end)
)

f([2, 3, 4], [2, 3])

```

This might differ slightly from the initial approach. Here, I defined various intermediate variables within the function. By defining the outputs of multiple functions, I can use the intermediate outputs as inputs for other functions. This way, my problem is solved. However, this approach relies on the order of function calculations, so it may be worth considering constructing a computational graph to determine the sequence.