# Turning Expressions into Functions

**URL:** <https://discourse.julialang.org/t/turning-expressions-into-functions/49558>\
**Category:** Performance\
**Tags:** metaprogramming\
**Created:** [November 4, 2020, 12:14am UTC](https://discourse.julialang.org/t/turning-expressions-into-functions/49558 "2020-11-04T00:14:56Z")\
**Posts on this page:** 18\
**Page:** 1

<div class="post-metadata">

**Author:** ![1ozturkbe](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/1ozturkbe/32/16461_2.png) [@1ozturkbe](https://discourse.julialang.org/u/1ozturkbe)\
**Post date:** [November 4, 2020, 12:14am UTC](https://discourse.julialang.org/t/turning-expressions-into-functions/49558/1 "2020-11-04T00:14:57Z")

</div>

I have the following Expression:

```julia
ex = :(sum(x[i] for i=1:4)- y[1] * y[2] + z)

```

which I would like to evaluate over some data x, y, z. I have tried doing direct substitution through symbolic manipulations, but it is abysmally slow (~0.2s per set of substitutions).  
It seems the right approach is to turn it into a function as [here](https://discourse.julialang.org/t/converting-an-expression-to-a-function/44736), but the application of that is somewhat limited.

Is there a method for converting `ex` into a function like the following?

```julia
f(x,y,z) = sum(x[i] for i=1:4)- y[1] * y[2] + z

```

Thanks in advance!

---

<div class="post-metadata">

**Author:** ![johnmyleswhite](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnmyleswhite/32/31_2.png) [@johnmyleswhite](https://discourse.julialang.org/u/johnmyleswhite)\
**Post date:** [November 4, 2020, 1:46am UTC](https://discourse.julialang.org/t/turning-expressions-into-functions/49558/2 "2020-11-04T01:46:55Z")

</div>

The standard questions apply here:

- How did you end up with expressions?
  - Are they based on user input?

- Can you handle the problem you’re trying to solve using macros?
  - If so, use macros.

- If you can only get access to the expressions at runtime in a way that rules out macros, you’ll likely need to use eval, but that is generally more challenging to do well and harder to make performant.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [November 4, 2020, 2:21am UTC](https://discourse.julialang.org/t/turning-expressions-into-functions/49558/3 "2020-11-04T02:21:19Z")

</div>

> **[GitHub - SciML/RuntimeGeneratedFunctions.jl: Functions generated at runtime...](https://github.com/SciML/RuntimeGeneratedFunctions.jl)**
>
> Functions generated at runtime without world-age issues or overhead - GitHub - SciML/RuntimeGeneratedFunctions.jl: Functions generated at runtime without world-age issues or overhead

---

<div class="post-metadata">

**Author:** ![brett\_knoss](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/brett_knoss/32/13050_2.png) [@brett\_knoss](https://discourse.julialang.org/u/brett_knoss)\
**Post date:** [November 4, 2020, 2:43am UTC](https://discourse.julialang.org/t/turning-expressions-into-functions/49558/4 "2020-11-04T02:43:37Z")

</div>

Is that used to parse symbolic expressions into functions, ie allow the output of ModelingToolkit to be used in Plots, or possibly another CAS?

---

<div class="post-metadata">

**Author:** ![moeddel](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/moeddel/32/18641_2.png) [@moeddel](https://discourse.julialang.org/u/moeddel)\
**Post date:** [November 4, 2020, 8:34am UTC](https://discourse.julialang.org/t/turning-expressions-into-functions/49558/5 "2020-11-04T08:34:09Z")

</div>

I had a similar problem with multi-variate polynomials. You can take a look into the package

[https://github.com/hofmannmartin/SphericalHarmonicExpansions.jl](https://github.com/hofmannmartin/SphericalHarmonicExpansions.jl)

There is a `fastfunc` macro and `fastfunc` function, which can turn any multi-variate polynomial into a fast evaluating function. An example for its usage can be found in [here](https://github.com/hofmannmartin/SphericalHarmonicExpansions.jl/blob/master/test/fastfunc.jl).

I hope this helps

---

<div class="post-metadata">

**Author:** ![1ozturkbe](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/1ozturkbe/32/16461_2.png) [@1ozturkbe](https://discourse.julialang.org/u/1ozturkbe)\
**Post date:** [November 4, 2020, 4:43pm UTC](https://discourse.julialang.org/t/turning-expressions-into-functions/49558/6 "2020-11-04T16:43:39Z")

</div>

Thanks @johnmyleswhite, here’s the best answer I have:

- I end up with the expressions through representations of constraints of optimization problems. They can be user-generated or read directly from a file. But in general, there are too many of these to be able to accurately and within reasonable time transcribe into functions.
- I’m not sure exactly how macros help here, but my knowledge of them is limited. If you can elaborate that would be appreciated.
- It is clear to me that eval is not an option, since performance is a significant issue.

@ChrisRackauckas thanks for the package link. I tried the following, but it throws an error, likely because there is the embedded sum:

```julia
ex = :((x, y, z) -> sum(x[i] for i=1:4)- y[1] * y[2] + z)
using RuntimeGeneratedFunctions
RuntimeGeneratedFunctions.init(@ __MODULE__ )
f = @RuntimeGeneratedFunction(ex)
f(ones(4), ones(2),5) #ERROR: The function body AST defined by this @generated function is not pure. 
                       #This likely means it contains a closure or comprehension.

```

Furthermore, it’s not clear to me how to actually specify the arguments given just the right hand side of the expression. I guess I have a lot to figure out about what is going on under the hood.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [November 4, 2020, 4:56pm UTC](https://discourse.julialang.org/t/turning-expressions-into-functions/49558/7 "2020-11-04T16:56:50Z")

</div>

> [@1ozturkbe](#):
>
> likely because there is the embedded sum:

It’s the generator expression `x[i] for i=1:4` ☹. Does GeneralizedGenerated.jl work here or does it have the same issue? I think it does have the same issue. I would just avoid the generator in the generated code.

---

<div class="post-metadata">

**Author:** ![1ozturkbe](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/1ozturkbe/32/16461_2.png) [@1ozturkbe](https://discourse.julialang.org/u/1ozturkbe)\
**Post date:** [November 4, 2020, 8:51pm UTC](https://discourse.julialang.org/t/turning-expressions-into-functions/49558/8 "2020-11-04T20:51:52Z")

</div>

GeneralizedGenerated.jl doesn’t work either… 😐 it seems like the only robust way to get around these issues is to go towards generating functions from the beginning!

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [November 4, 2020, 8:55pm UTC](https://discourse.julialang.org/t/turning-expressions-into-functions/49558/9 "2020-11-04T20:55:11Z")

</div>

Or use `eval`. Eval won’t have overhead but you just have to watch out for world-age issues.

---

<div class="post-metadata">

**Author:** ![GraceMabele](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gracemabele/32/21360_2.png) [@GraceMabele](https://discourse.julialang.org/u/GraceMabele)\
**Post date:** [April 6, 2021, 6:03am UTC](https://discourse.julialang.org/t/turning-expressions-into-functions/49558/10 "2021-04-06T06:03:51Z")

</div>

Hi

I would like to find out if you eventually obtained a solution as I am currently having the same problem and cannot seem to find a solution.

---

<div class="post-metadata">

**Author:** ![simeonschaub](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/simeonschaub/32/216566_2.png) [@simeonschaub](https://discourse.julialang.org/u/simeonschaub)\
**Post date:** [April 6, 2021, 6:31am UTC](https://discourse.julialang.org/t/turning-expressions-into-functions/49558/11 "2021-04-06T06:31:19Z")

</div>

If you are on 1.7, you could instead write this as:

```julia
using Base.Experimental: @opaque
ex = :((x, y, z) -> sum(Base.Generator(@opaque(i -> x[i]), 1:4)) - y[1] * y[2] + z)

```

and then use RuntimeGeneratedFunctions on `ex` as described above.

---

<div class="post-metadata">

**Author:** ![JeffreySarnoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeffreysarnoff/32/1980_2.png) [@JeffreySarnoff](https://discourse.julialang.org/u/JeffreySarnoff)\
**Post date:** [April 6, 2021, 7:18am UTC](https://discourse.julialang.org/t/turning-expressions-into-functions/49558/12 "2021-04-06T07:18:09Z")

</div>

following the directions

```julia
f(ones(4), ones(2),5) 
 #ERROR: The function body AST defined by this @generated function is not pure.
8.0

```

is the error spurious or else is there a fix?

---

<div class="post-metadata">

**Author:** ![simeonschaub](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/simeonschaub/32/216566_2.png) [@simeonschaub](https://discourse.julialang.org/u/simeonschaub)\
**Post date:** [April 6, 2021, 7:20am UTC](https://discourse.julialang.org/t/turning-expressions-into-functions/49558/13 "2021-04-06T07:20:37Z")

</div>

Can you try this in a fresh session? I don’t get any errors:

```julia
julia> using Base.Experimental: @opaque

julia> ex = :((x, y, z) -> sum(Base.Generator(@opaque(i -> x[i]), 1:4)) - y[1] * y[2] + z)
:((x, y, z)->begin
          #= REPL[2]:1 =#
          (sum(Base.Generator(#= REPL[2]:1 =# @opaque((i->begin
                                      #= REPL[2]:1 =#
                                      x[i]
                                  end)), 1:4)) - y[1] * y[2]) + z
      end)

julia> using RuntimeGeneratedFunctions

julia> RuntimeGeneratedFunctions.init(@ __MODULE__ )

julia> f = @RuntimeGeneratedFunction(ex)
RuntimeGeneratedFunction(#=in Main=#, #=using Main=#, :((x, y, z)->begin
          #= REPL[2]:1 =#
          (sum(Base.Generator(#= REPL[2]:1 =# @opaque((i->begin
                                      #= REPL[2]:1 =#
                                      x[i]
                                  end)), 1:4)) - y[1] * y[2]) + z
      end))

julia> f(ones(4), ones(2),5)
8.0

```

---

<div class="post-metadata">

**Author:** ![GraceMabele](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gracemabele/32/21360_2.png) [@GraceMabele](https://discourse.julialang.org/u/GraceMabele)\
**Post date:** [April 6, 2021, 7:28am UTC](https://discourse.julialang.org/t/turning-expressions-into-functions/49558/14 "2021-04-06T07:28:38Z")

</div>

Does it work on 1.6 as 1.7 does not seem to be available

---

<div class="post-metadata">

**Author:** ![simeonschaub](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/simeonschaub/32/216566_2.png) [@simeonschaub](https://discourse.julialang.org/u/simeonschaub)\
**Post date:** [April 6, 2021, 7:32am UTC](https://discourse.julialang.org/t/turning-expressions-into-functions/49558/15 "2021-04-06T07:32:11Z")

</div>

No, you will need 1.7. You can download the nightly builds [here](https://julialang.org/downloads/nightlies/), but be warned that it is still in active development, so you might encounter some bugs.

---

<div class="post-metadata">

**Author:** ![JeffreySarnoff](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jeffreysarnoff/32/1980_2.png) [@JeffreySarnoff](https://discourse.julialang.org/u/JeffreySarnoff)\
**Post date:** [April 6, 2021, 8:18am UTC](https://discourse.julialang.org/t/turning-expressions-into-functions/49558/16 "2021-04-06T08:18:54Z")

</div>

Thank you. A clean REPL worked without error.

---

<div class="post-metadata">

**Author:** ![misha\_mikhasenko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/misha_mikhasenko/32/5060_2.png) [@misha\_mikhasenko](https://discourse.julialang.org/u/misha_mikhasenko)\
**Post date:** [April 12, 2021, 10:55am UTC](https://discourse.julialang.org/t/turning-expressions-into-functions/49558/17 "2021-04-12T10:55:48Z")

</div>

Could someone clarify please, what problems of `eval` approach are?  
I had a similar problem recently solved with eval

```julia
using InteractiveUtils

function give_xyz_header_eval(expr)
    fexpr = quote
        f(x,y,z) = $expr
    end
    eval(fexpr)
end

# tests
evaluated_f = give_xyz_header_eval(:(x^2+y^2+z^2-2x*y))
@code_warntype evaluated_f(1,2,3) # looks fine

evaluated_fsum = give_xyz_header_eval(:(sum(x[i] for i=1:4)- y[1] * y[2] + z))
@code_warntype evaluated_fsum(ones(4),ones(2),3) # looks fine

```

---

<div class="post-metadata">

**Author:** ![simeonschaub](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/simeonschaub/32/216566_2.png) [@simeonschaub](https://discourse.julialang.org/u/simeonschaub)\
**Post date:** [April 12, 2021, 12:41pm UTC](https://discourse.julialang.org/t/turning-expressions-into-functions/49558/18 "2021-04-12T12:41:16Z")

</div>

`eval` is fine for use at the toplevel.

However, if you try to call a function inside the same function you are defining that function in with `eval` like, e.g.:

```julia
function foo(a)
    evaluated_f = give_xyz_header_eval(:(x^2+y^2+z^2-2x*y))
    return evaluated_f(a, a, a)
end

```

, you will run into worldage errors when trying to run `foo`.

A workaround to this is to invoke `evaluated_f` with `Base.invokelatest(evaluated_f, a, a, a)`, but this has significant overhead and the result of `evaluated_f` will always be inferred as `Any`, so `foo` can never be type stable.

`@RuntimeGeneratedFunction` works around this by using some hacks involving generated functions to achieve the same thing without the overhead and without giving up on type inference.