# Best performance for eval of expressions with parameters

I try to achieve better/best performance for eval(expression).

Up to now I found 5 more or less different possibilities (in respect to my code logic) to eval an expression. It’s best, to see it in code. The following MWE can be directly copy&pasted into a REPL. It reflects my real projects structure in a minimal way.

• the `try...catch...end` is needed, because the real expressions e can result in errors, e.g. `(-0.5) ^ 0.9`
• `f3::Function` is not a real option, its just for comparisson of performance
• in my real project, I have several hundreds of different `e::Expr` and have to evaluate them in a loop over hundreds of thousands. I am trying some kind of evolution of expressions (genetic programming).
``````#copy&paste into REPL:

using BenchmarkTools

module M

mutable struct Ef
e::Expr
f1::Function
f2::Function
f3::Function
function Ef()
e = :(a + b)
f1 = Base.eval( :( (a,b)->\$e ) )
f2 = Core.eval( M , :( (a,b)->\$e ) )
f3 = (a,b)->a+b
new(e,f1,f2,f3)
end
end

function eval_global(a_ef::Array{Ef,1})
r=0
for ef in a_ef
global a=3
global b=5
try
r += Core.eval(M, ef.e)
catch e
end
end
return r
end

function eval_let(a_ef::Array{Ef,1})
r=0
for ef in a_ef
val1=3
val2=5
ex=ef.e
try
r += @eval begin
let
a=\$val1
b=\$val2
\$ex
end
end
catch e
end
end
return r
end

interpolate_from_dict(ex::Expr, dict) = Expr(ex.head, interpolate_from_dict.(ex.args, Ref(dict))...)
interpolate_from_dict(ex::Symbol, dict) = get(dict, ex, ex)
interpolate_from_dict(ex::Any, dict) = ex
function eval_interpolate(a_ef::Array{Ef,1})
r=0
for ef in a_ef
try
r += Base.eval(interpolate_from_dict(ef.e,Dict( :a => 3, :b => 5 )))
catch e
end
end
return r
end

function eval_f1(a_ef::Array{Ef,1})
r=0
for ef in a_ef
try
r += ef.f1(3,5)
catch e
end
end
return r
end

function eval_f2(a_ef::Array{Ef,1})
r=0
for ef in a_ef
try
r += ef.f2(3,5)
catch e
end
end
return r
end

function eval_f3(a_ef::Array{Ef,1})
r=0
for ef in a_ef
try
r += ef.f3(3,5)
catch e
end
end
return r
end

function f(a,b)
try
a+b
catch e
0
end
end

end

a_ef=[ M.Ef() for i in 1:1000 ]

#1
@btime M.eval_global(\$a_ef)

#2
@btime M.eval_let(\$a_ef)

#3
@btime M.eval_interpolate(\$a_ef)

#4
@btime M.eval_f1(\$a_ef)

#5
@btime M.eval_f2(\$a_ef)

#6
@btime M.eval_f3(\$a_ef)

#7
@btime sum( [ M.f(3,5) for i in 1:1000 ] )
``````

The fastest way to do this calculation is #5 (perhaps equal with #4), which is not surprising, as the eval step is done only once and the loop is just calling the resulting function.

The question now is:

Is there a (much) faster way to do this calculation on expressions?

I don’t really understand why #6 is about 4 times faster than #5 and #4.

#7 is of order 10^3 faster than #4 and #5 without `try...catch...end`. Adding this as it is now in the above MWE it is only about 10 times faster than #4 and #5.

So `try...catch...end` seems to cost quite something.

Still: is it possible to get near #7 with something similar to #4 and #5?

Maybe implement a custom bytecode and interpreter rather than using `Expr`?

1 Like

Thanks, this would probably going fastest, but would actually be a project on itself for me, as I don’t have any experience on this.

I found something else (after playing around with `@generated`) here
https://discourse.julialang.org/t/a-method-to-remove-the-use-of-runtime-eval-and-invokelatest-as-well-as-support-closures-in-generated-functions-come-with-an-implemented-proptype/

which is condensed into this package

``````#copy&paste into REPL:
using BenchmarkTools

module M
using GeneralizedGenerated

mutable struct Ef
e::Expr
f1::Function
f2
function Ef()
e = :(a + b)
f1 = (a,b)->a+b
f2 = mk_function( :(  (a, b) -> \$e ) )
new(e,f1,f2)
end
end

function eval_f1(a_ef::Array{Ef,1})
r=0
for ef in a_ef
r += ef.f1(3,5)
end
return r
end

function eval_f2(a_ef::Array{Ef,1})
r=0
for ef in a_ef
r += ef.f2(3,5)
end
return r
end

end

a_ef=[ M.Ef() for i in 1:1000 ]

``````

Performance of the generated function `f2` is as fast as the static version `f1`:

``````julia> @btime M.eval_f1(\$a_ef)
38.800 μs (937 allocations: 14.64 KiB)
8000

julia> @btime M.eval_f2(\$a_ef)
39.199 μs (937 allocations: 14.64 KiB)
8000
``````
1 Like