Storing evaled function inside a struct

General Usage
21

Just use a closure to construct whatever function you want out of cost and other functions.

f = let a=2, d=somefunction(a)
    x -> cost(a, x[1], a*x[1], d)
end

and just pass f to your optimization function.

It would return a function (of the unknown parameters) to be optimized, as in my x -> example above.

This is called a higher-order function, and is generally a vastly superior (faster, more flexible, more composable) alternative to calling eval on expressions constructed at runtime.

6 Likes
22

Yes, basically a list of

q = (:a, [:b, :c, :d], :(a = b + c + d)) 
# if you have all of q[2] you can calculate q[1] with the expression in q[3]

which increases the list of parameters I know or can calculate. I iterate this until everything is either known, calculable or is free (= an optimization variable).

This sounds great but I’m still unsure of how to do this.
From your example

f = let a=2, d=somefunction(a)
    x -> cost(a, x[1], a*x[1], d)
end

assuming I know the let needs to look just like this, how do I create it? I only know I’ll need somefunction and not otherfunction at runtime so I can’t just write it down. Somewhere in there the dependencies would need to be resolved (and from that I don’t know what to return other than expressions).

The only option I can think of right now would be along the lines of

function resolve(config)
    if config == example_1
        a = (x) -> 2
        b = (x) -> x[1]
        c = (x) -> 2 * x[1]
        d = (x) -> 2 / 2
        return a, b, c, d
    else config == example_2
        a = (x) -> 2 * x[1]
        b = (x) -> 5
        c = (x) -> ((x) -> 2 * x[1])(x) * 5  # not sure yet how to put one value inside another
        d = (x) -> x[1]
        return a, b, c, d
    end
end

f = let
    fa, fb, fc, fd = resolve(config)
    x -> cost(fa(x), fb(x), fc(x), fd(x))
end

During the resolving I’d look up the functions I have written out here.
Did you mean something like this?

23

Two points, don’t use expression if you can use function and use metaprogramming to save typing rather than to do anything else.

What you want to do is clearly to save typing, so you should treat it like that. Don’t go straight to evaling an exprerssion.

You should think about how you’d write this without metaprogramming first. I can think of at least two ways to do this.

  1. You can just do

    a = 2  # the const value I want
d = 1  # == a / 2;  I get this value by evaluating this equation while generating the expressions
b = x[1]
c = a * b
cost(a, b, c, d)

  2. Or you can also do your dependency tracking with

    context = initialize_context_with_initially_known_values()
cost(get_value(context, :a), get_value(context, :b), get_value(context, :c), get_value(context, :d))

    and you’ll keep track of whethter a variable is computed in context and call the computing functions to calculate them if necessary. The computing functions are implemented in a similar way, say the one for c would be get_value(context, :a) * get_value(context, :b).

Now you know how you are going to do this without metaprogramming, in both cases you’ll be typing a function to do the computation, your only job left is how to type that functiton with the fewest characters. In another word, you want a macro like @cost_function cost(a, b, c, d) that expand to one of the function definitions above. Such a macro should be pretty easy to write. You pretty much already have it for the first case (hard one) and it is trivial for the second one.

4 Likes
24

First of all, thanks for taking the time to help me.

So, if I understand correctly, option 1 would be:
Do the same thing as I’m doing now but instead of putting the expressions in a function and evaling that, turn it into a macro which returns the expressions in a function which can be called after running the macro.

I found option 2 harder to grasp (probably because I’m pretty entrenched in thinking about the problem in one way) so I wrote some mockup code as I understood the idea:

mutable struct Var
    store::Int
    val::Float64    # fixed value
    func::Function  # to be calculated from other values
    idx::Int        # index in the array of optimization values
end

mutable struct Context
    a::Var
    b::Var
    ...
end

function initialize_context_with_initially_known_values(...)
    c = Context()
    var = Var()
    var.store = 1
    var.func = calc_a_from_d
    # where calc_a_from_d(context) = get_value(context, :d) * 2
    c.a = var
    # and the same with the other variables, while figuring out
    # dependencies
    return c
end

function get_value(context, n, x)
    var = getfield(context, n)
    if var.store == 0
        return var.val
    elseif var.store == 1
        return var.func(context)
    else
        return x[var.idx]
    end
end

Is this in the ballpark?

25

You can cache the value you calculate, which you can also use o store the pre defined constant instead of using an index into an array and you can also use a Bool instead of Int for store.

26

I’m not sure I understand. Indexing into the array would be needed to get the values from optimization (optimize(x -> cost(get_value(context, :a, x) ...))) and because of that not all the values calculated with the function will only depend on constants or have been looked up already.

27

Well, you can store the value whatever way you want but I’m saying that you’ve already assigned names to them and without more constraints you don’t need to give them index at the same time.

28

Alright, I tried to implement option 2; it should give me the possibility to create contexts dynamically (as opposed to a macro), which would be an upside over what is currently working with eval in the REPL.

However, I have a lot of trouble getting everything working and typestable. At the moment I am thinking that it is inherently impossible to get it to be typestable because getfield(context, n).func (from my mockup above) could be a function of type calc_a_from_d or something_else – until the code runs there is no way to know.

If I try to split it up like

function get_value(context, n, x)
    var = getfield(context, n)
    _get(var, x)
end

function _get(var{T, F}, x) where {T, F}  # I added types to val and func
    if var.store == 0
        return var.val
    elseif var.store == 1
        return var.func(context, x)
    else
        return x[var.idx]
    end
end

I get _get inferred and fast, but not get_value (which makes sense to me if my reasoning above is correct).

Is it correct though? Or is there a way to work around this?

29

Correct. It’s not meant to be type stable. Though if fyou know the type you can always assert it.