# Implement a convenient interface to keep some parameter fix for optimization

Hi all,

First I have to apologize because I could not write this questions as concise as I’d like to.

My goal is to implement an interface for optimisation/inference where the user can keep certain parameters fix (i.e. not inferred). Additionally, I would like that the user has only to deal with parameters in the form of `NameTuple`s.

### Example

To make it more concrete assume the following example:

``````using TransformVariables

# 1) Define model

# The transform is used for flattening and transformation of the parameters
tt = as((
a = asℝ₊,
b = asℝ₊,
w = UnitVector(5)
))

function model(p::NamedTuple, x)
p.a + p.b * x'*p.w
end

model(v::AbstractArray, x) = model(tt(v), x)

# 2) User can call model with NamedTuple

p = (a = 11.1,
b = 2.1,
w = [0.2, 0.2, 0.2, 0.2, 0.2])

x = ones(5)

model(p, x)

# 3) ... and optimizer/sampler routines can call model with V ∈ R^n

v = randn(6)
model(v, x)

# 4) we can define a function to fit the model

function fit_model(p_init::NamedTuple)

v_init = inverse(tt, p_init)          # convert into vector
v_optimal = optim(v_init, model, ...) # some optimizer routine, that returns a vector
p_optimal = tt(v_optimal)             # user gets back a nice tuple

return p_optimal
end

# The user provides inital parameters in
# the "natural" form, i.e. as NamedTuple
p_init = (a = 3.1,
b = 5.1,
w = [0.2, 0.2, 0.2, 0.2, 0.2])

fit_model(p_init)  # ... and gets a Tuple back. All good so far!
``````

### Desired user interface

Now, how can I write a extended version `fit_model2` that can be used as follows:

``````# The idea is that the user has to use only the NamedTuple format, i.e.

p_init = (a = 3.1,
b = 5.1,
w = [0.2, 0.2, 0.2, 0.2, 0.2])

p_keep_fix = (a = false,
b = true,
w = [false, false,  true, true, false])

fit_model2(p_init, p_keep_fix)  # <- how to implement this?
``````

### First attempt

I think the structure would look a bit like this. I’m stuck how I can write the ‘splice’ function

``````function fit_model2(p_init::NamedTuple, p_keep_fix::NamedTuple)
# 1) Transform the initial parameters to R^n.
v_init = inverse(tt, p_init) |> filter(is_not_fixed)

# 2) transform the fixed values to R^n
v_fix = inverse(tt,  p_keep_fix) |> filter(is_fixed)

# 3) Splice the fixed and not fixed parameters together in the
# right order. It is not so easy to figure how many parameters are to be optimized,
# as it depends also on the transformation! For example, p.w is
# UnitVector so no parameters must be estimated if either 4 or 5
# values of p.w are fixed.

# Then build function to optimize:
function fopt(v)
v' = splice(v, v_fix)
model(v')
end

# 4) Optimize as usual
v_opt = optim(v_init, fopt)

# 5) combine fiexed and optimized parameters
v_opt' = splice(v_opt, v_fix)

# 6) user get a Named Tuple back
return tt(v_opt')

end
``````

It seems an common and not very difficult problem. Looking forward to any ideas!

The normal practice would be to capture fixed parameters with closures. e.g. `optimize(p -> somefunction(p, a, b), pinit, ...)` where `a` and `b` are fixed parameters.

(In NLopt, you can set upper and lower bounds on each variable, so another way to keep a parameter fixed is to set upper bound = lower bound.)

1 Like

Thanks! That’s my goal here (see `fopt` in my first attempt).

But my question is how I can construct such a closure automatically, given the Tuples `p_init` and `p_keep_fix`.