How to convert a string into an expression?

metaprogramming
#1

Hello,
I want to generate lines of code automatically, using a list of field names as input.
The following code doesn’t throw an error, but it also doesn’t produce the desired result, because the eval function needs an expression as argument and not a string.
How can I make it work?

#!/usr/bin/env julia
using CSV

# create dummy csv file for testing
csv =
"""
time, pos_x, pos_y, pos_z
0.0, 0.0, 0.0, 0.0
0.1, 1.0, 2.0, 3.0
0.2, 1.5, 3.0, 4.0
"""
f = open("test.csv", "w")
write(f, csv)
close(f)

# read the csv file into a DataFrame
data = CSV.read("test.csv")

# dummy function for testing
function resample(a,b)
    return b
end

# define the fields for code generation
fields = "pos_x","pos_y","pos_z"

# preparations
const slice = 1:2
const times = data[:time]
const time_slice = times[slice]

# create constant, global arrays in a loop
for field in fields
    line = ("const $field = resample(time_slice, data[:$field][slice])")
    println(line)
    eval(line)
end

#=
# The following code should be generated and executed
const pos_x = resample(time_slice, data[:pos_x][slice])
const pos_y = resample(time_slice, data[:pos_y][slice])
const pos_z = resample(time_slice, data[:pos_z][slice])
=#
Any hints appreciated!
#2

In general, parse (or Meta.parse) takes string and converts it into expression that can be evaluated. But in your case I don’t see why you can’t define fields as symbols and generate expression directly, e.g. for one field:

field = :pos_x
line = :(const $field = resample(time_slice, data[:($field)][slice]))
#3

Thanks a lot for your suggestion, but it doesn’t work:
Code:

# define the fields for code generation
fields = :pos_x, :pos_y, :pos_z

# preparations
const slice = 1:2
const times = data[:time]
const time_slice = times[slice]

# create constant, global arrays in a loop
for field in fields
    println(field)
    line = :(const $field = resample(time_slice, data[:($field)][slice]))
    println(line)
    eval(line)
end

Output:

pos_x
const pos_x = resample(time_slice, (data[$(Expr(:quote, :($(Expr(:$, :field)))))])[slice])
ERROR: LoadError: UndefVarError: field not defined
Stacktrace:
 [1] eval(::Module, ::Any) at ./boot.jl:235
 [2] macro expansion at /media/msata/uavtalk/scripts/meta_test.jl:41 [inlined]
 [3] anonymous at ./<missing>:?
 [4] include_from_node1(::String) at ./loading.jl:576
 [5] include(::String) at ./sysimg.jl:14
while loading /media/msata/uavtalk/scripts/meta_test.jl, in expression starting on line 37

Any idea?

#4

I think I need to use the function parse in my case. The following code works:

# define the fields for code generation
fields = :pos_x, :pos_y, :pos_z

# preparations
const slice = 1:2
const times = data[:time]
const time_slice = times[slice]

# create constant, global arrays in a loop
for field in fields
    line = parse("const $field = resample(time_slice, data[:$field][slice])")
    eval(line)
end
#5

That should never be the case. Period.

:(const $field = resample(time_slice, data[$(QuoteNode(field))][slice]))

#6

Or using @eval:

for field in fields
    @eval const $field = resample(time_slice, data[$(QuoteNode(field))][slice])
end
#7

What is QuoteNode doing? I found no documentation in the help.

#8

It encapsulates a symbol so that interpolating it will give you back the symbol instead of the value bound to the variable with that name.

#9

You can also use Meta.quot for this.

#10

Thanks for the explanations!

And what is the disadvantage of using parse?

#11

It’s fragile and insecure. It’s very easy to accidentally get something that is not what you intended. Think SQL injection but with arbitrary Julia code. Even if it’s not externally exposed, it can still accidentally do things you did not intend.

#12

If you can, I’d stick with the suggestion to use @eval, instead of building up Exprs directly, at least at the moment with the transition between v0.6.x and v0.7, as there have been a number of changes to the AST, that have been a bit of a pain when trying to write code with Expr that works both on v0.6.2 and master (I’m not saying that the changes aren’t good, and the AST should be fairly stable after the v1.0 release, it’s only an issue for probably very few people right at the moment)