I have a “model” that require many parameters and I am struggling to keep a reference to all the paramters in the various subcalls. I would like instead to define a julia file to host the “default” parameters, itself a parameter with the default default parameters
I would like to keep the parameter file(s) as a julia file (instead of, let’s say, a json or toml file) because in these files I would like to make simple computations, typically simple calibrations of my model parameters from some more “raw” data.
So I have a function mymodel that I would like to call with:
mymodel(defaults="somefile.jl", par2=1.0, par4=10) where par2 and par4 override those defined in somefile.jl
mymodel would then be defined with: function mymodel(;defaults="default_of_defaults.jl",kwargs...)
Does it look reasonable ?
Can I just include(defaults) in the body of my function and then build a struct with the parameters to pass it around ?
However, this means I would have multiple include as I call the function multiple times…
Maybe my approach is too simplistic, but I prefer to use simply named tuples and a preprocess function. That seems cleaner than an abstract framework around everything.
For example:
toml file defines raw parameters, load as named tuple
preprocess function gets raw parameters, inserts default values of from the toml input missing params and then does preprocessing.
final parameter object is again a named type, e.g. no need to redefine structs etc.
A simplistic example looks like this:
function preprocess(p_raw)
p_def = (x = 10,)
p = (p_def..., p_raw...)
y = p.x^2
p = (p..., y)
return p
end
There is a bit lot of unpacking and merges of names tuples, but that way it remains vanilla Julia code.
If you like defining your parameters with julia code but otherwise keep a file that mostly consists of top-level x = ... expressions, you could also try a macro that reads in the expressions from your settings file and collects all the top-level assigned variables into a NamedTuple or so. The settings file is read in at compile time so there’s no eval or include overhead.
If you have a file settings.jl:
a = 1 + 2
b = 3
c = strip(" hello $b ")
d = let
x = 1
y = 2
x + y
end
Then you could use the macro workflow like this, for example:
function globals_to_namedtuple(expr)
syms = Symbol[]
for arg in expr.args
if arg isa Expr && arg.head == :(=)
push!(syms, arg.args[1])
end
end
quote
let
$(expr)
(; $(syms...),)
end
end
end
macro settings(path::String)
globals_to_namedtuple(Meta.parseall(read(path, String)))
end
@settings "settings.jl"
# (a = 3, b = 3, c = "hello 3", d = 3)
I used to do something like this but found that using named tuples in the preprocessing function leads to huge compile times as the number of parameters grows, especially when doing many merges. Now I do the preprocessing using ordered dictionaries Symbol=>Any and convert to a name tuple at the end.
I don’t know if this could help but, in my modeling API, I chose to define a parameter container (mutable structure with the default parameter values) for each category of model function. And I built a function (recovkw.jl) that loads the container (as well eventually replaces default values by those set in kwargs) when the function is called. A simple example is given below
## PLSR model function
Base.@kwdef mutable struct ParPlsr
nlv::Union{Int, Vector{Int}, UnitRange} = 1
scal::Bool = false
end
function plskern!(X::Matrix, Y::Union{Matrix, BitMatrix}, weights::Weight; kwargs...)
## load defaults parameters values and eventually replaces some of
## them by those set in kwargs by the user
par = recovkw(ParPlsr, kwargs).par
## End
...
end
and
model = plskern!(nlv = 15) # parameters inside the model will be nlv = 15, scal = false
