Building an anonymous callback function on the fly

[Niall]

Hi. This (I think) is probably a question about meta-programming. I’m writing a System Dynamics (Stock and Flow) wrapper around the DynamicalSystems package. In a method named build!(), I pull together the various bits of stock-and-flow information that the user wants to build into a DynamicalSystems model, in order to put those bits together into a dynamical_rule function that the constructor CoupledODEs() requires as its first argument. The end of build!() looks like this:

	dynamical_rule! = (du,u,p,n) -> (
		p[:] = eval.(p_vals);
		du[:] = sum.(map(x->eval.(x),du_vals));
		nothing
	)

#	sdm.ode_model = CoupledODEs(dynamical_rule!, u_vals, p_vals)	# sdm is ready to run!
	(dynamical_rule!,p_vals,du_vals)
end     # of build!()

I have currently commented out the call to CoupledODEs() and instead am returning the three items of interest like this:

julia> dynamical_rule!,p_vals,du_vals = SystemDynamics.build!(niall)
(Main.SystemDynamics.var"#3#7"{Vector{Vector{Union{Float64, Expr}}}, Vector{Union{Float64, Expr}}}(Vector{Union{Float64, Expr}}[[:(p[1] * u[1])]], Union{Float64, Expr}[0.3]), Union{Float64, Expr}[0.3], Vector{Union{Float64, Expr}}[[:(p[1] * u[1])]])

julia> p_vals
1-element Vector{Union{Float64, Expr}}:
 0.3

julia> du_vals
1-element Vector{Vector{Union{Float64, Expr}}}:
 [:(p[1] * u[1])]

julia> dynamical_rule!([1.0],[2.0],[3.0],4.0)
ERROR: UndefVarError: `p` not defined
Stacktrace:
 [1] top-level scope
   @ none:1

So I have two questions really…

• First, I have clearly misunderstood how to link the argument p of dynamical_rule!() to the code that I have constructed in the statement
  du[:] = sum.(map(x->eval.(x),du_vals));
  When dynamical_rule!() is called, this should (in my understanding) evaluate to
  du[:] = p[1] * u[1],
  however julia says that p is not defined. So clearly I’ve got things wrong somewhere.
• Second, I’m currently getting tangled up with lots of Union{Float64,Expr} objects that I don’t really want. What I really want are Exprs that evaluate to Float64 values, and which may in fact on occasion simply be numeric literals. Unfortunately, however, expressions such as :(0.3) are immediately translated into a Float64 and so are never recognised as Exprs.

If anyone happens to be into this sort of stuff, I’d very much appreciate a helping hand.

Thanks!

[mkitti]

It would be helpful if the examples you provided were actually executable, but I can try.

julia> e = :(2p)
:(2p)

julia> f(e,p) = eval(e)
f (generic function with 1 method)

julia> f(e,5)
ERROR: UndefVarError: `p` not defined

help?> eval
search: eval evalpoly evalfile @eval @evalpoly bytesavailable

  eval(expr)

  Evaluate an expression in the global scope of the
  containing module. Every Module (except those defined with
  baremodule) has its own 1-argument definition of eval,
  which evaluates expressions in that module.

As you can see eval operates in the global scope, not in the function’s local scope.

The solution is to use more anonymous functions.

julia> e = p->2p
#3 (generic function with 1 method)

julia> f(e,p) = e(p)
f (generic function with 1 method)

julia> f(e,5)
10

Using anonymous functions will also address your literals problem.

julia> e2 = p->0.3
#5 (generic function with 1 method)

julia> f(e2,5)
0.3

Also, let me introduce you to the functor which is a struct that can be called like a function.

julia> mutable struct MyFunctor
           a::Int
           b::Float64
       end

julia> (mf::MyFunctor)(x) = mf.a*x + mf.b

julia> e = MyFunctor(3, 2.7)
MyFunctor(3, 2.7)

julia> e(9)
29.7

julia> e.a = 300
300

julia> e(9)
2702.7

[Benny]

@mkitti

As you can see eval operates in the global scope, not in the function’s local scope.

To address the common followup question, there is no way around this in any language. By the time a function call executes, information on variables and scopes has been obscured or discarded during lowering and compiling; a variable doesn’t even exist, having been wrangled into possibly several locations in a stack frame (zero locations, if the compiler finds it’s not needed at runtime). It’s just too late to parse, compile, and execute code at runtime as if it belonged in the function’s scope to begin with. In Julia, the function’s Expr is your window to transform its code, whether that’s done at parse-time in a macro, compile-time in a @generated function (with severe limitations), or at runtime in the global scope or in a function.

What mkitti suggests is basically making your expressions into functions of their own, and you share data between different functions by passing in arguments and returning values. To apply his hint directly to your example, you might make du_vals = [(p, u) -> (p[1] * u[1])]. If you really need to start from Expr, you could process it into an expression outputting a function, then call eval to evaluate it in the global scope and return the output function to the call scope.

PS just for style, you can use begin ... end blocks instead of ( ...; ...; ...), to avoid the need for line-ending semicolons and visual similarity to tuples. Since that expression is also in an anonymous function, you can also make a multi-line anonymous function with function (args...) ... end by omitting the function name.

[Niall]

I knew there was a good reason for posting my question, even though it was poorly formulated. Thank you very much, both of you! You’ve both packed your responses so full of useful information that I’m pretty sure I can turn it all into an elegant solution.

Oh, and I now see that the issue about eval() using the module’s global scope was actually obvious from the documentation - I just hadn’t thought it through sufficiently.

Thanks again,
Niall.