Evaluating a SyntaxTree.genfun within a function

Specific Domains
,
1

Im trying to evaluate a string as a function within a function. I have made good progress:

using SyntaxTree
function Evaluate(some_function::String ; variables = [:x])
    g = genfun(Meta.parse(some_function), variables)
    return g
end

This allows me to write Evaluate(“some function”)(x)
which is good, but if I want to evaluate this inside another function, for example:

function foo(some_function::String ; variables = [:x])
     x = 1
     return Evaluate(some_function)(x)
end

I get an Error

MethodError: no method matching (::getfield(SyntaxTree, Symbol("##137#138")))(::Float64)
The applicable method may be too new: running in world age 26255, while current world is 26256.
Closest candidates are:
  #137(::Any) at /Users/joe/.julia/packages/SyntaxTree/Adq4Y/src/SyntaxTree.jl:144 (method too new to be called from this world context.)

Stacktrace:
 [1] h(::Float64) at ./In[113]:1
 [2] WaveFunctionNumerov(::String, ::Float64, ::Int64, ::Float64, ::Float64, ::Float64) at ./In[113]:24
 [3] top-level scope at In[114]:52

Why is this a problem in the first place and how can I get around it?

the evaluation really needs to be within a function

1 Like
2

First thing first: why?
I ask because I see people try to do this a lot, and it has never been the right answer.

That being said, it looks like SyntaxTree offers genlatest which should work, since it calls Base.invokelatest, getting around the world age issue. Again, this is probably the wrong approach to the problem, with plenty of negative performance and runtime consequences, so I urge you to reconsider.

2 Likes
3

Hi, i am the creator of SyntaxTree and I recommend using Julia expressions for metaptogramming if possible instead of parsing strings, as @tomerarnon mentioned it is not so good to use strings for this in general.

Indeed, the purpose of genlatest is to get around the world age issues.

3 Likes
4

The problem is that genfun most likely uses eval and eval always runs at runtime. Therefore, when the compiler looks at foo, eval has not been run yet, so it doesn’t have any way of knowing what to do with the call Evaluate(some_function)(x). This is what it means for g to only be defined in a newer world age. You can use Base.invokelatest, to work around this, but that has quite a big performance penalty. An alternative might be to use RuntimeGeneratedFunctions.jl, which can build functions from expressions at runtime without the use of eval, but is a bit hacky.

1 Like
5

The idea is that when running the julia script, the script will ask for an input, (in the form of a String or Expr) and then run the program from there making the program more dynamic and diverse.

I will try the genlatest, although I’d be more than happy to hear a more effective solution if you have one?

6

Why would this make a difference? Surely parsing a String into an Expr is effectively the same thing as just taking an Expr as input?

7

True, in this case using a string or an expr would be the same (and suffer from the same limitations).
If you intend to create a function from a user inputted string, consider separating the Evaluate function into two separate operations in your script. First create the genfun function (which will no longer need to be genlatest, as a result of this split) and then pass the created function on to another function to do the work. This will ensure type stability and will therefore be more performant.

g = Evaluate(user_input) # type unstable, but who cares?
foo(g) # foo is specialized to typeof(g) now!
2 Likes