What is the best way to avoid world age problem in this case?

General Usage
,
1

I am writing a package that uses a YML file as the user interface. Some parameters in the YML file are numbers which are easy to handle. A few parameters need to be functions. I ask the user to put those functions in some file and refer to that file and the function names within that file in the YML file. For examle:

functions.jl contains

f1(a) = a^2

and the YML file contains:

function name: f1
function file: functions.jl

In my package, I read and use the parameters using Base.invokelatest:


function read_params(param_file)
  d = YAML.load_file(param_file)
  include(d["function file"])
  return eval(Symbol(d["function name"]))
end

function use_params(param_file)
  func = read_params(param_file)
  return Base.invokelatest(func, 12)
end

Is this the best way to solve this problem? What other alternatives would you suggest?
With this method, I face problems when multiple replicates in parallel.

2

First question: why do you need functions selected in a YML file, and do you really need that? What are you trying to do? whats the big picture here?

Having user-defined input interacting with the julia method table in ways that world age matters normally ends badly.

2 Likes
3

Well, the YML file is where the user defines all the parameters. A simple human readable text file. Some of the parameters are functions -rather than being fixed scalars or vectors- and they cannot be put in the YML file. Do you know a better alternative?

4

Back up a little: what are you trying to do at all. What is the user trying to do. Why is a parameter a function. What field even is this!?

I’m trying to avoid X Y problems because this is a classic X Y problem kind of question!

1 Like
5

Sure. This is a package to run a certain agent-based model. The parameters define the agents (age, phenotypes, distribution, etc) and the environment (its size, resources, etc). Some parameters can change stochastically over time, for example, the available food across time and space, hence, they cannot fixed vectors. Food availability, given time and region, is a user-defined function.
Does this answer your question?

1 Like
6

Yes: what you are talking about is similar to problems in my work (with DynamicGrids.jl).

The easiest solution is to not use YML. You are already using a julia script anyway - functions.jl. What do you get from using both a julia script and a YML file?

For variable sets of parameters to run parallel with the same code, you can use a csv (with only paremeters, never functions). I wrote ModelParameters.jl for loading CSV data into arbitrary model structures to do things like run in parallel with different params.

The other trick is for user defined code is define structs instead of functions, and use the same function that dispatches on them. Then the time of loading the script does not matter, the object type matters. Then they can also include parameters (as fields in the sctruct) that you can vary from a CSV instead of using multiple different functions as you seem to be.

If you are using the same function name with different code but no differences in dispatch, you are going to create some hair pulling issues for users. Try to avoid situations where the state of the method table determines the functionality. Your solution should work or error, and never be silently wrong.

3 Likes
7

Great advice! Thanks

8

All the advice above is great. Just to solve the problem as it is directly asked, you could define a dictionary of options for the function parameter like

Dict( "first option" => func1,
        "second option" => func2)

And then the user could chose their function by key in the yaml file.

Another thing to consider is keeping everything in Julia. Base.@kwdef lets you make a struct that could act basically like a yaml file:

@Base.kwdef struct ModelParameters
    environment_size = (100, 100)
    people_number = 10_000
    food_at_start = 5_000
    resource_change_function = myresourcefunction
end

I think it’s about as human-readable as a yaml file.

2 Likes
9

Interesting solution! Thank you!

1 Like
10

I tried your suggested solution, but I still get the world age problem. I now use a dictionary for the user to input the model parameters.

params.jl:

f1(a::Int) = a^2

parameters = Dict(
  :p1 => f1
)

main.jl:

module Model

function read_params(param_file)
  include(param_file)
  return parameters
end

function use_params(param_file)
  parameters = read_params(param_file)
  return parameters[:p1](2)
end

end

Running the program:

include("main.jl")
using Main.Model
Model.use_params("params.jl")

ERROR: MethodError: no method matching f1(::Int64)
The applicable method may be too new: running in world age 32419, while current world is 32420.
Closest candidates are:
  f1(::Int64) at E:\projects\mwe_include\params.jl:1 (method too new to be called from this world context.)      
Stacktrace:
 [1] use_params(param_file::String)
   @ Main.Model E:\mwe_include\main.jl:10
 [2] top-level scope
   @ REPL[3]:1
11

No, the point is that you should just create a callable type and change the parameters each time, not the function type.

struct MyF{T}
  a::T
end

(f::MyF)(x) = a + x

f = MyF(2) # create f with a=2
f(3) # 2 + 3 = 5
1 Like
12

Thanks. But what should be passed in the parameter file?
I tried the following and it didn’t work.

params.jl:

struct MyF{T}
  a::T
end

(f::MyF)() = f.a^2

parameters = Dict(
  :p1 => MyF
)

main.jl:

function read_params(param_file, x)
  include(param_file)
  f = parameters[:p1](x)
  return f()
end
13

Just the parameters.

1 Like