# Handling solver parameters elegantly in Julia

**URL:** <https://discourse.julialang.org/t/handling-solver-parameters-elegantly-in-julia/5269>\
**Category:** General Usage\
**Tags:** question\
**Created:** [August 8, 2017, 1:24am UTC](https://discourse.julialang.org/t/handling-solver-parameters-elegantly-in-julia/5269 "2017-08-08T01:24:09Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![juliohm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juliohm/32/215266_2.png) [@juliohm](https://discourse.julialang.org/u/juliohm)\
**Post date:** [August 8, 2017, 1:24am UTC](https://discourse.julialang.org/t/handling-solver-parameters-elegantly-in-julia/5269/1 "2017-08-08T01:24:09Z")

</div>

Consider a simple solver type:

```julia
using Parameters: @with_kw

@with_kw struct Param
  mean = 0.
  variance = 1.
end

struct Solver
  params::Dict{Symbol,Param}

  function Solver(params...)
    new(Dict(params...))
  end
end

```

Based on this interface, the user can construct solvers like the following:

```julia
Solver(:var1=>Param(mean=2.), :var2=>Param(variance=3.))

```

The idea is that for each variable of the problem one can specify a set of parameters that will trigger a different method. The code is working fine, but I guess there is a cleaner way of achieving the same result without this ugly `Param` type in front of the parameters?

```julia
Solver(:var1=>(mean=2.), :var2=>(variance=3.), :var3=>(mean=1.,var=2.))

```

Do you have suggestions on how to refactor the code to make it look nicer? The actual implementation can be found here: [https://github.com/juliohm/GeoStats.jl/blob/master/src/kriging\_solver.jl](https://github.com/juliohm/GeoStats.jl/blob/master/src/kriging_solver.jl)

More generally, is there any effort in a package to generalize this process? Are macros useful here?

---

<div class="post-metadata">

**Author:** ![juliohm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juliohm/32/215266_2.png) [@juliohm](https://discourse.julialang.org/u/juliohm)\
**Post date:** [August 8, 2017, 5:31am UTC](https://discourse.julialang.org/t/handling-solver-parameters-elegantly-in-julia/5269/2 "2017-08-08T05:31:09Z")

</div>

Something along the lines of [GetPot for C++](http://getpot.sourceforge.net) would be interesting. Reading algorithm parameters from files or dealing with them in the code directly.

---

<div class="post-metadata">

**Author:** ![juliohm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juliohm/32/215266_2.png) [@juliohm](https://discourse.julialang.org/u/juliohm)\
**Post date:** [August 9, 2017, 12:35am UTC](https://discourse.julialang.org/t/handling-solver-parameters-elegantly-in-julia/5269/3 "2017-08-09T00:35:48Z")

</div>

@mauro3 since you are the Parameters master 🙂 do you have suggestions on how to improve the interface?

---

<div class="post-metadata">

**Author:** ![tshort](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tshort/32/43_2.png) [@tshort](https://discourse.julialang.org/u/tshort)\
**Post date:** [August 9, 2017, 1:49am UTC](https://discourse.julialang.org/t/handling-solver-parameters-elegantly-in-julia/5269/4 "2017-08-09T01:49:10Z")

</div>

You could use [NamedTuples.jl](https://github.com/blackrock/NamedTuples.jl) to get something closer to your “cleaner” option. When named tuples come to Julia 0.7, the syntax will be even nicer.

You could also use a TOML string/file for this type of thing. That’s more like your GetPot option. (It’s also coming to more use in Julia as part of Pkg3.)

---

<div class="post-metadata">

**Author:** ![juliohm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juliohm/32/215266_2.png) [@juliohm](https://discourse.julialang.org/u/juliohm)\
**Post date:** [August 9, 2017, 3:34am UTC](https://discourse.julialang.org/t/handling-solver-parameters-elegantly-in-julia/5269/5 "2017-08-09T03:34:50Z")

</div>

Thank you @tshort NamedTuples.jl seem like a very good fit. From what I understood, the interface would look something like:

```julia
Solver(:var1 => @NT(mean=1.), :var2 => @NT(variance=3.))

```

What is the syntax for Julia v0.7 that you mentioned?

---

<div class="post-metadata">

**Author:** ![juliohm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juliohm/32/215266_2.png) [@juliohm](https://discourse.julialang.org/u/juliohm)\
**Post date:** [August 9, 2017, 4:51am UTC](https://discourse.julialang.org/t/handling-solver-parameters-elegantly-in-julia/5269/6 "2017-08-09T04:51:28Z")

</div>

I tried to set some default solver parameters an ran into this issue:

[https://github.com/blackrock/NamedTuples.jl/issues/27](https://github.com/blackrock/NamedTuples.jl/issues/27)

Will wait until Julia v0.7 is out.

---

<div class="post-metadata">

**Author:** ![mauro3](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mauro3/32/292_2.png) [@mauro3](https://discourse.julialang.org/u/mauro3)\
**Post date:** [August 9, 2017, 8:59am UTC](https://discourse.julialang.org/t/handling-solver-parameters-elegantly-in-julia/5269/7 "2017-08-09T08:59:10Z")

</div>

> [@juliohm](#):
>
> Parameters master 🙂

Haha! But no, I have no answers only a few comments:

The deeper you nest your options, the more organized they are but also the more painful to deal with.

With scripting languages there is rarely a need to use separate configuration files: just use a script instead, saves you the hassle to deal with another package and the user the hassle of learning/context-switching to another mini-language.

Have a look at how DiffEq handles options, mostly through kwargs I think.

---

<div class="post-metadata">

**Author:** ![tshort](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tshort/32/43_2.png) [@tshort](https://discourse.julialang.org/u/tshort)\
**Post date:** [August 9, 2017, 10:41am UTC](https://discourse.julialang.org/t/handling-solver-parameters-elegantly-in-julia/5269/8 "2017-08-09T10:41:46Z")

</div>

I’d use keyword args for Solver. With 0.7 syntax, that’d be:

```julia
Solver(var1 = (mean=1.), var2 = (variance=3.))

```

You may need an extra comma in there.

[https://github.com/JuliaLang/julia/pull/22194](https://github.com/JuliaLang/julia/pull/22194)

You might also define a shorter name for Param. You can do this now:

```julia
Solver(var1 = P(mean=1.), var2 = P(variance=3.))

```

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [August 9, 2017, 1:45pm UTC](https://discourse.julialang.org/t/handling-solver-parameters-elegantly-in-julia/5269/9 "2017-08-09T13:45:11Z")

</div>

> [@mauro3](#):
>
> Have a look at how DiffEq handles options, mostly through kwargs I think.

Keyword args to the solver. Keyword arguments currently have an overhead, but that should get fixed soon enough so I wouldn’t worry about it. The cost isn’t too high though, so if it’s just in the setup phase you won’t even notice it. If you do it via splatting positional args and then assuming the args are in order, you can at least make it like

```julia
Solver(Params(mean=2.), Params(variance=3.), Params(mean=1.,var=2.))

```

which is a little nicer.

---

<div class="post-metadata">

**Author:** ![juliohm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juliohm/32/215266_2.png) [@juliohm](https://discourse.julialang.org/u/juliohm)\
**Post date:** [August 9, 2017, 3:00pm UTC](https://discourse.julialang.org/t/handling-solver-parameters-elegantly-in-julia/5269/10 "2017-08-09T15:00:54Z")

</div>

I thought of using positional arguments and assume order, but the interface looks more obscure. I like explicit pairs var1=\>param1, var2=\>…, but I may change this point of view in the future.

---

<div class="post-metadata">

**Author:** ![juliohm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juliohm/32/215266_2.png) [@juliohm](https://discourse.julialang.org/u/juliohm)\
**Post date:** [August 21, 2017, 5:13am UTC](https://discourse.julialang.org/t/handling-solver-parameters-elegantly-in-julia/5269/11 "2017-08-21T05:13:58Z")

</div>

@tshort, I am back to this issue after a while. I am giving NamedTuple a try, but couldn’t make the outer constructor below work:

```julia
using Parameters: @with_kw

@with_kw struct Param
 mean = 0.
 variance = 1.
end

struct Solver
 params::Dict{Symbol,Param}

 # inner ctor to be called from outside
 Solver(params::Dict{Symbol,Param}) = new(params)
end

# outer ctor with named tuples
function Solver(params...)
 dict = Dict{Symbol,Param}()
 for (varname, varparams) in params
   push!(dict, varname => Param(varparams...)) # doesn't work
 end

 Kriging(dict)
end

```

Could you please give a hand? The goal is to have the Solver working with named tuples as you described:

```julia
Solver(:var1 => @NT(mean=1), :var2 => @NT(variance=2.))

```

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [August 21, 2017, 11:26pm UTC](https://discourse.julialang.org/t/handling-solver-parameters-elegantly-in-julia/5269/12 "2017-08-21T23:26:17Z")

</div>

Could you more explicitly say why normal keywords arguments don’t work here.

---

<div class="post-metadata">

**Author:** ![juliohm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juliohm/32/215266_2.png) [@juliohm](https://discourse.julialang.org/u/juliohm)\
**Post date:** [August 21, 2017, 11:36pm UTC](https://discourse.julialang.org/t/handling-solver-parameters-elegantly-in-julia/5269/13 "2017-08-21T23:36:00Z")

</div>

Hi @kristoffer.carlsson, it is because I want to give the user a very explicit API. I have an example of it in the README: [https://github.com/juliohm/GeoStats.jl#examples](https://github.com/juliohm/GeoStats.jl#examples)

If the user loads a spreadsheet with column names, I want him to be explicit about the variables by name, not by position.

In that example, the user would be able to call the solver with:

```julia
Kriging(:precipitation => @NT(mean=1., variogram=GaussianVariogram()))

```

With more variables one would list them explicitly, no matter the order they were defined in the problem:

```julia
Kriging(
  :precipitation => @NT(mean=1.),
  :co2_concentration => @NT(variogram=GaussianVariogram()),
...
)

```

This would become even more clear in Julia v0.7 without the @NT boilerplate.

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [August 22, 2017, 12:20am UTC](https://discourse.julialang.org/t/handling-solver-parameters-elegantly-in-julia/5269/14 "2017-08-22T00:20:05Z")

</div>

> [@juliohm](#):
>
> I want him to be explicit about the variables by name, not by position.

Keyword arguments are by name and not position so not sure what you mean here.

If you need to give many key-values for each variable then you could just use a Dict for each variable?

---

<div class="post-metadata">

**Author:** ![juliohm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juliohm/32/215266_2.png) [@juliohm](https://discourse.julialang.org/u/juliohm)\
**Post date:** [August 22, 2017, 12:31am UTC](https://discourse.julialang.org/t/handling-solver-parameters-elegantly-in-julia/5269/15 "2017-08-22T00:31:32Z")

</div>

The problem is that I have many variables in the problem. So if I say mean=1, what variable it refers to? That is why keyword arguments don’t solve the problem. I need to have pairs `variable_name => (mean=1)` to disambiguate instead of assuming that the user will remember the order of the variables when he defined the EstimationProblem.

To be more precise:

```julia
problem = EstimationProblem(data, domain, [:var1,:var2])
solve(problem, Kriging(:var1 => (params...), :var2 => (params...))

```

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [August 22, 2017, 12:45am UTC](https://discourse.julialang.org/t/handling-solver-parameters-elegantly-in-julia/5269/16 "2017-08-22T00:45:36Z")

</div>

You could just use a dictionary instead of a named tuple. If you really want to use named tuples you could convert the named tuple to a dictionary inside the `Solver` function:

```julia
julia> a = @NT(mean = 0.0, variance = 0.0)
(mean = 0.0, variance = 0.0)

julia> d = Dict(k=>v for (k,v) in zip(fieldnames(typeof(a)), a))
Dict{Symbol,Float64} with 2 entries:
  :mean => 0.0
  :variance => 0.0

julia> Param(d...)

```

Or `@kwdef` could, in addition, generate a constructor for a named tuple.

---

<div class="post-metadata">

**Author:** ![juliohm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juliohm/32/215266_2.png) [@juliohm](https://discourse.julialang.org/u/juliohm)\
**Post date:** [August 22, 2017, 1:20am UTC](https://discourse.julialang.org/t/handling-solver-parameters-elegantly-in-julia/5269/17 "2017-08-22T01:20:26Z")

</div>

That is exactly my question, how to write an outer constructor that converts named tuples to dictionaries. I have tried it in my previous comment, but failed. I will try with your code snippet now.

---

<div class="post-metadata">

**Author:** ![juliohm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juliohm/32/215266_2.png) [@juliohm](https://discourse.julialang.org/u/juliohm)\
**Post date:** [August 22, 2017, 1:34am UTC](https://discourse.julialang.org/t/handling-solver-parameters-elegantly-in-julia/5269/18 "2017-08-22T01:34:27Z")

</div>

@kristoffer.carlsson this is my second attempt, now with your code snippet, it didn’t work unfortunately:

```julia
using Parameters: @with_kw
using NamedTuples

@with_kw struct Param
 mean = 0.
 variance = 1.
end

struct Solver
 params::Dict{Symbol,Param}

 # inner ctor to be called from outside
 Solver(params::Dict{Symbol,Param}) = new(params)
end

# outer ctor with named tuples
function Solver(params...)
 # build Dict for inner ctor
 dict = Dict{Symbol,Param}()

 for (varname, varparams) in params
   # varparams is a NamedTuple
   d = Dict(k => v for (k,v) in zip(fieldnames(typeof(varparams)), varparams))

   push!(dict, varname => Param(d...))
 end

 Solver(dict) # call inner ctor
end

```

The conversion from NamedTuple to Dict works, but what I really need is to pass the NamedTuple entries as keyword arguments to the Param type, which takes care of all the checks like valid parameter names, types, etc.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [August 22, 2017, 5:49am UTC](https://discourse.julialang.org/t/handling-solver-parameters-elegantly-in-julia/5269/19 "2017-08-22T05:49:29Z")

</div>

> [@juliohm](#):
>
> didn’t work

Posting an MWE (which includes the way you want to call this function) would increase the probability that you get help on this.

I am not sure why you need an inner constructor for `Solver`, the default one would do the same as above.

---

<div class="post-metadata">

**Author:** ![juliohm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juliohm/32/215266_2.png) [@juliohm](https://discourse.julialang.org/u/juliohm)\
**Post date:** [August 22, 2017, 6:37am UTC](https://discourse.julialang.org/t/handling-solver-parameters-elegantly-in-julia/5269/20 "2017-08-22T06:37:48Z")

</div>

@Tamas_Papp, sorry, I thought the MWE was clear from my previous comments. I am trying to have a solver type that I can create with:

```julia
solver = Solver(
  :var1 => @NT(mean=1.),
  :var2 => @NT(variogram=GaussianVariogram())
)

```

Each pair in this list would be converted to a pair of Symbol and Param, where Param is defined above. The Param type has keyword constructors from the Parameters.jl package. This way I am enforcing meaningful parameters for each variable even though the user types arbitrary NamedTuples like @NT(not\_valid=2.).

My issue is in defining the outer constructor that does this conversion Dict{Symbol,NamedTuple} to Dict{Symbol,Param}. This outer constructor is vararg.

Please let me know if is still not clear, all the code from my previous comment is correct, except for the outer constructor, which doesn’t compile.

The reason I am doing this now is because I hope that in Julia v0.7, the @NT will go away and then the syntax will be super clean:

```julia
Solver(
  :var1 => (mean=1.),
  :var2 => (variogram=GaussianVariogram())
)

```

[Next page](https://discourse.julialang.org/t/handling-solver-parameters-elegantly-in-julia/5269.md?page=2)