How to avoid evaluation into the global space when the expression is not know in advance

General Usage
,
1 
input = Dict("x"=>ones(10),"y"=>2*ones(10),"z"=>3*ones(10))

expr_list = DataFrame(var=["v1","v2","v3"],expr = ["x*y","y/z","x+y+z"])

output = Dict()

I have a dictionary of input storing the values of some variables x,y,z. Output variables v1,v2,v3 are calculated based on their definition in the expr_list.

I’ve heard that eval should be avoided in such calculations, but the examples I found assume that you already know the expressions so you can do something to avoid eval. But if the expr_list is not know in advance what should I do?

Here is my bad solution using eval. This enters x,y,z into the global space, which I don’t want.


for (key,value) in input

    eval(Meta.parse("$key = $value"))

end

for i in eachrow(expr_list)

    output[i.var] = eval(Meta.parse("@. $(i.expr)"))

end
2

You cannot avoid evaluating into global scope if you want to parse strings and evaluate them. Julias semantics “only” allow for functions to see the result of eval when they return back to global scope. It’s a key aspect of the language, which allows it to be compiled & fast while still being a dynamic language. You can find an analysis of why that is here (it’s a little longer, but well worth the read).

What are you trying to evaluate/interpret in the first place? How did you arrive at a solution involving eval? Maybe there’s a better abstraction for what you’re originally trying to do, we could help you find that abstraction if you tell us a little more about it.

3 Likes
3

The input are results from previous calculations generated by the modeler. The expr_list is supplied by the user using “txt” or “xlsx” files, which the modeler doesn’t know in advance. The modeler should calculate the new variables based on the expr_list and generate an output “txt” or “xlsx” file.

I don’t want x,y,z to enter the global space in this calculation because they might already have values in the global space. Or that these names might be used again later.

4

Don’t store expressions as strings. Store them as functions.

function_list = [ (x,y,z) -> x*y, (x,y,z) -> y/z, (x,y,z) -> x+y+z ]
input = (ones(10), 2*ones(10), 3*ones(10))
output = [ f.(input...) for f in function_list ]
3 Likes
5

The problems are that

  1. The expr_list is supplied by as string expressions in a “txt/xlsx” file. Can you convert a string to a function without eval.
  2. In reality there are hundreds of variables in the input list x,y,z,a,b,c,d,....., I’m only showing three here as an examples. To write anonymous functions like (x,y,z,a,b,c,d...) - > ... is unwieldy.
6

No.

It really sounds like you’d want whoever is using your code to write julia code - why not let them do that in the first place? Instead of you trying to parse & run their code, you could structure your code as a library for them to use & include in their code.

1 Like
7

I’m not sure what you mean. I feel this should be a very simple task, just processing existing data into new output. So unless you manually write the expressions and turn into functions etc like @stevengj did, there’s not way to avoid eval ?

8

You’re asking about taking julia code (are you?) in form of strings and running that as code. If that’s not what eval does, I don’t know what is.

Like I said, it kind of sounds like you’d want people to write julia code in the first place. That code could either use your code as a library or you include or eval it. If we’re not talking about julia code in form of strings, what are we talking about exactly?

9

I think the simplest solution to what you want is to make the user write the functions as:

(x,y) -> x*y
(x,y,z) -> x + y + z

julia> string1 = "(x,y) -> x*y"
"(x,y) -> x*y"

julia> f1 = eval(Meta.parse(string1))
#7 (generic function with 1 method)

julia> f1(2,2)
4

Then you can eval those expressions, which will define functions (in the global scope, but is always the case, AFAIK).

It become more complicated if you really don’t want the user to have understand that (x,y)->x*y is a function that receives x and y and returns x*y. You could write your own “list your variables here” table in the format you find convenient. If you wan’t to avoid the user to even listing which are the variables, than you have write some sort of parsers yourself for the strings, to get them.

That can work, and I see the potential uses of that, particularly for a simple user interface, some educational software, etc. For a more involved computation all that parsing of strings will become cumbersome and very error prone.

10

That’s much more complex than what I’m looking for. Maybe my description was confusing.

My aim is very simple: Suppose my friend give me two txt files, one input containing a data matrix of old variables, and one expr_list containing string expression of calculating new variables. (I don’t see how these files are generated matters, whether by Julia or not, assuming they are formatted correctly). My question is how do I calculate the output without eval?

11

the only (practical) way to turn arbitrary string into Julia function is through eval. If you know the expressions are polynomial etcl., you can parse the coefficients without eval, but from your original questions it sounds like the content of expr_list can be anything.

1 Like
12

What they are saying is that you can do this:

file: “data.jl”

data = [ 
           1,
           2,
 ]

file: functions.jl

functions = [ 
           x -> 2*x,
           x -> 3*x
]

then do:

julia> include("./data.jl")

julia> include("./functions.jl")

julia> for (f,d) in zip(functions,data)
          println(f(d))
       end
2
6

That is not that different from what you want, and you don’t need any complicated evaluation. Just need to give the input as to julia arrays with the proper syntax in the original files.

1 Like
13

Yes. They can be any combinations of elementary functions.

14

I see. But I can’t ask people to write function.jl for me, I only have a txt or excel file containing the string expressions.

15

That literally sounds like they’re basically writing julia code already, except they don’t want to/can’t write proper functions and are instead limited to a subset of julia that only works on the top level…?

If you don’t want to clobber the global scope, there’s no way around some kind of eval into a temporary module.

Also, treating non-julia code like julia code will probably run into weird edge cases in the future.

2 Likes
16

So: either they can write the functions as

(x,y) -> x*y

and then use eval on those (that is fine for that purpose). Or, if you really only can expect the user to write

x*y

You need to parse that string to understand which are the variables (are they very limited? is it just checking if x, y and z are present?), and turn that into a string as:

(x,y) -> x*y

But how general you want the input to be will define how complicated this parsing will be.

1 Like
17

The advantage of this is that you generate anonymous functions that do not conflict with anything else in the the global namespace?

1 Like
18

Exactly, they are just functions.

1 Like
19

Be aware that those functions will still be kept around for the duration of the running julia process and that the functions array proposed here would also hang around. There’s also no caching between similar functions, since they’d all be new anonymous functions and thus julia will have to compile them again and again.

2 Likes
20

That seems to be the most simple solution.

1 Like