Metaprogramming using eval and struct: BUG getting UndefVarError: not defined

I am trying to read parameters from a .csv file for which the name of parameter and its values are given in the .csv file.

For that I am trying to simply evaluate a simple expression involving putting the value of the parameter in a struct, but I get error.
I will be delighted if you could provide me a solution to this simple problem?

# Defining structure
mutable struct PARAM
     θs  :: Vector{Float64}
end
θs = zeros(Float64, 1)
hydro = PARAM(θs)

Evaluate =" hydro.θs[1]=0.5"
Evaluate_Parse = Meta.parse(Evaluate)
eval(Evaluate_Parse)

. Nevertheless, I get the following error message
ERROR: LoadError: UndefVarError: hydro not defined

It runs well if just paste your code in REPL, do you have the definition of hydro and the execution of eval in different modules?

Why don’t you just use a Dict and avoid metaprogramming?

2 Likes

There seem to be a bug since you are correct the code works well in REPL but fails in a function.
Yes I have the definition of hydro and the execution of eval in a different modules

eval can only see variables in the global scope, it cannot see local variables.

Thanks for shedding light to the problem, how can I make mutable struct into global scope?

It’s not about mutable sutrct PARAM, it’s about the variable hydro.

  • hydro must be global in some Module
  • eval must be ensured to evaluate an expression in that Module.

In the code below, hydro is a global var in module T, and T.eval is used to ensure that the parsed expression are evaluated in module T.

julia> module T
       mutable struct PARAM
            θs  :: Vector{Float64}
       end
       θs = zeros(Float64, 1)
       hydro = PARAM(θs)
       end
Main.T

julia> Evaluate =" hydro.θs[1]=0.5"
" hydro.θs[1]=0.5"

julia> Evaluate_Parse = Meta.parse(Evaluate)
:(hydro.θs[1] = 0.5)

julia> eval(Evaluate_Parse)
ERROR: UndefVarError: hydro not defined
Stacktrace:
 [1] top-level scope at none:1
 [2] eval at ./boot.jl:331 [inlined]
 [3] eval(::Expr) at ./client.jl:451
 [4] top-level scope at REPL[4]:1

julia> T.eval(Evaluate_Parse)
0.5

julia> T.hydro
Main.T.PARAM([0.5])

julia>

1 Like

Generally it is advisable to avoid using eval. From your example I do not see why you need eval at all as directly using hydro.θs[1] = 0.5 without using eval would work.

If your use case requires names that you will only know after you load the CSV I would suggest storing the data in either a Dict or a DataFrame depending on your use case. Both of these allow you to index by name.

3 Likes

Thanks for your great help and your time. Unfortunately when I introduced your recommendation into the complex program, I was not able to make it work, I am still getting the same error. I think that I would need more clarity on the meaning of Main.T ?

T is just the name of the module that KDr2 is using to hold onto your global variable. All variables you want to change/assign/update/touch with eval would need to be globals there — and you need to ensure that you’re explicitly evaling into that module.

But I really think you’ll want to change your angle of attack here — using eval like this isn’t very sustainable or maintainable. The other alternatives proposed here will likely make your life much easier. Another option would be a file format like TOML that can work nicely for what I think you’re trying to do.

3 Likes

PROBLEM RESOLVED :smile:

I want to thank you for inviting me to take a different approach which I solved as follow:

mutable struct PARAM
     θs  :: Vector{Float64}
end

setfield!(hydro, Symbol(θs), Parameter)

I get the value of the parameter θs from from a .csv file. This effort is to perform multistep optimisation.

1 Like

@JosephPollacco Note that this is the same as

hydro.θs = parameter

if I understand correctly

You are correct that it is the same as:

hydro.θs = parameter

The advantage of

setfield!(hydro, Symbol("θs"), Parameter)

Is that you can perform Metaprogramming by avoiding eval. I can now read the symbol directly from a csv file and put it automatically into the structure hydro

But that only works if you have already defined a struct with the correct field name beforehand.

You are correct, it is the downside of the method. :disappointed:
If you have a method to create struct on the fly that will be great. My challenge is that some parameters are vectors.

Just use a Dict, as was already mentioned several times:

d = Dict()

d["θs"] = 3
d["hello"] = [4, 5]

Of course this will be more efficient if all the values are known to be of a certain type:

d = Dict{String, Int64}()
d["a"] = 3
d["b"] = -17
1 Like

Or you can use a NamedTuple, which is similar to a struct.
NamedTupleTools.jl helps with that.

1 Like

Thanks for the suggestion of using Dict

My understanding is that one of the limitation of Dict is that one cannot use Vectors. “θs” is a parameter describing the maximum saturated soil moisture of a layer in a soil. I have many soil layers, so my understanding is that Dict might not be the best solution for using Vectors, but I might be wrong. For e.g. this may not be possible:

d["θs[1]"] = 1
d["θs[2]"] = 2

Just do d["θs"]=[1,2], ie store the vector as the value.

1 Like

Did you actually try to run that code? It works fine.


> d["θs[1]"] = 1
1

> d["θs[2]"] = 2
2

> d
Dict{Any,Any} with 2 entries:
  "θs[2]" => 2
  "θs[1]" => 1

But that’s not what you really want (probably).

I think you want


> d = Dict()
Dict{Any,Any} with 0 entries

> d["θs"] = [3, 4, 5]
3-element Array{Int64,1}:
 3
 4
 5

> d["θs"][1]
3

> d["θs"][2]
4

d["θs"] is now a Vector whose elements you can extract in the usual way.

2 Likes