Best performance for eval of expressions with parameters

General Usage Performance
1

I try to achieve better/best performance for eval(expression).

Up to now I found 5 more or less different possibilities (in respect to my code logic) to eval an expression. It’s best, to see it in code. The following MWE can be directly copy&pasted into a REPL. It reflects my real projects structure in a minimal way.
Some notes about it:

  • the try...catch...end is needed, because the real expressions e can result in errors, e.g. (-0.5) ^ 0.9
  • f3::Function is not a real option, its just for comparisson of performance
  • in my real project, I have several hundreds of different e::Expr and have to evaluate them in a loop over hundreds of thousands. I am trying some kind of evolution of expressions (genetic programming).
#copy&paste into REPL:

using BenchmarkTools

module M

mutable struct Ef
    e::Expr
    f1::Function
    f2::Function
    f3::Function
    function Ef()
        e = :(a + b)
        f1 = Base.eval( :( (a,b)->$e ) )
        f2 = Core.eval( M , :( (a,b)->$e ) )
        f3 = (a,b)->a+b
        new(e,f1,f2,f3)
    end
end

function eval_global(a_ef::Array{Ef,1})
    r=0
    for ef in a_ef
        global a=3
        global b=5
        try
            r += Core.eval(M, ef.e)
        catch e
        end
    end
    return r
end

function eval_let(a_ef::Array{Ef,1})
    r=0
    for ef in a_ef
        val1=3
        val2=5
        ex=ef.e
        try
            r += @eval begin
                    let
                        a=$val1
                        b=$val2
                        $ex
                    end
                end
        catch e
        end
    end
    return r
end

interpolate_from_dict(ex::Expr, dict) = Expr(ex.head, interpolate_from_dict.(ex.args, Ref(dict))...)
interpolate_from_dict(ex::Symbol, dict) = get(dict, ex, ex)
interpolate_from_dict(ex::Any, dict) = ex
function eval_interpolate(a_ef::Array{Ef,1})
    r=0
    for ef in a_ef
        try
            r += Base.eval(interpolate_from_dict(ef.e,Dict( :a => 3, :b => 5 )))
        catch e
        end
    end
    return r
end

function eval_f1(a_ef::Array{Ef,1})
    r=0
    for ef in a_ef
        try
            r += ef.f1(3,5)
        catch e
        end
    end
    return r
end

function eval_f2(a_ef::Array{Ef,1})
    r=0
    for ef in a_ef
        try
            r += ef.f2(3,5)
        catch e
        end
    end
    return r
end

function eval_f3(a_ef::Array{Ef,1})
    r=0
    for ef in a_ef
        try
            r += ef.f3(3,5)
        catch e
        end
    end
    return r
end

function f(a,b)
    try
        a+b
    catch e
       0
    end
end

end

a_ef=[ M.Ef() for i in 1:1000 ]

#1
@btime M.eval_global($a_ef)

#2
@btime M.eval_let($a_ef)

#3
@btime M.eval_interpolate($a_ef)

#4
@btime M.eval_f1($a_ef)

#5
@btime M.eval_f2($a_ef)

#6
@btime M.eval_f3($a_ef)

#7
@btime sum( [ M.f(3,5) for i in 1:1000 ] )

The fastest way to do this calculation is #5 (perhaps equal with #4), which is not surprising, as the eval step is done only once and the loop is just calling the resulting function.

The question now is:

Is there a (much) faster way to do this calculation on expressions?

I don’t really understand why #6 is about 4 times faster than #5 and #4.

#7 is of order 10^3 faster than #4 and #5 without try...catch...end. Adding this as it is now in the above MWE it is only about 10 times faster than #4 and #5.

So try...catch...end seems to cost quite something.

Still: is it possible to get near #7 with something similar to #4 and #5?

2

Maybe implement a custom bytecode and interpreter rather than using Expr?

1 Like
3

Thanks, this would probably going fastest, but would actually be a project on itself for me, as I don’t have any experience on this.

I found something else (after playing around with @generated) here
https://discourse.julialang.org/t/a-method-to-remove-the-use-of-runtime-eval-and-invokelatest-as-well-as-support-closures-in-generated-functions-come-with-an-implemented-proptype/

which is condensed into this package
https://github.com/thautwarm/GeneralizedGenerated.jl

#copy&paste into REPL:
using BenchmarkTools

module M
using GeneralizedGenerated

mutable struct Ef
    e::Expr
    f1::Function
    f2
    function Ef()
        e = :(a + b)
        f1 = (a,b)->a+b
        f2 = mk_function( :(  (a, b) -> $e ) )
        new(e,f1,f2)
    end
end

function eval_f1(a_ef::Array{Ef,1})
    r=0
    for ef in a_ef
        r += ef.f1(3,5)
    end
    return r
end

function eval_f2(a_ef::Array{Ef,1})
    r=0
    for ef in a_ef
        r += ef.f2(3,5)
    end
    return r
end

end

a_ef=[ M.Ef() for i in 1:1000 ]

Performance of the generated function f2 is as fast as the static version f1:

julia> @btime M.eval_f1($a_ef)
  38.800 μs (937 allocations: 14.64 KiB)
8000

julia> @btime M.eval_f2($a_ef)
  39.199 μs (937 allocations: 14.64 KiB)
8000
1 Like