Implement a convenient interface to keep some parameter fix for optimization

General Usage
,
1

Hi all,

First I have to apologize because I could not write this questions as concise as I’d like to.

My goal is to implement an interface for optimisation/inference where the user can keep certain parameters fix (i.e. not inferred). Additionally, I would like that the user has only to deal with parameters in the form of NameTuples.

Example

To make it more concrete assume the following example:

using TransformVariables

# 1) Define model

# The transform is used for flattening and transformation of the parameters
tt = as((
    a = asℝ₊,
    b = asℝ₊,
    w = UnitVector(5)
))

function model(p::NamedTuple, x)
    p.a + p.b * x'*p.w
end

model(v::AbstractArray, x) = model(tt(v), x)


# 2) User can call model with NamedTuple

p = (a = 11.1,
     b = 2.1,
     w = [0.2, 0.2, 0.2, 0.2, 0.2])

x = ones(5)

model(p, x)


# 3) ... and optimizer/sampler routines can call model with V ∈ R^n

v = randn(6)
model(v, x)


# 4) we can define a function to fit the model

function fit_model(p_init::NamedTuple)

    v_init = inverse(tt, p_init)          # convert into vector
    v_optimal = optim(v_init, model, ...) # some optimizer routine, that returns a vector
    p_optimal = tt(v_optimal)             # user gets back a nice tuple

    return p_optimal
end


# The user provides inital parameters in
# the "natural" form, i.e. as NamedTuple
p_init = (a = 3.1,
          b = 5.1,
          w = [0.2, 0.2, 0.2, 0.2, 0.2])

fit_model(p_init)  # ... and gets a Tuple back. All good so far!

Desired user interface

Now, how can I write a extended version fit_model2 that can be used as follows:

# The idea is that the user has to use only the NamedTuple format, i.e.

p_init = (a = 3.1,
          b = 5.1,
          w = [0.2, 0.2, 0.2, 0.2, 0.2])

p_keep_fix = (a = false,
              b = true,
              w = [false, false,  true, true, false])

fit_model2(p_init, p_keep_fix)  # <- how to implement this?

First attempt

I think the structure would look a bit like this. I’m stuck how I can write the ‘splice’ function

function fit_model2(p_init::NamedTuple, p_keep_fix::NamedTuple)
    # 1) Transform the initial parameters to R^n.
    v_init = inverse(tt, p_init) |> filter(is_not_fixed)

    # 2) transform the fixed values to R^n
    v_fix = inverse(tt,  p_keep_fix) |> filter(is_fixed)


    # 3) Splice the fixed and not fixed parameters together in the
    # right order. It is not so easy to figure how many parameters are to be optimized,
    # as it depends also on the transformation! For example, p.w is
    # UnitVector so no parameters must be estimated if either 4 or 5
    # values of p.w are fixed.

    # Then build function to optimize:
    function fopt(v)
        v' = splice(v, v_fix)
        model(v')
    end

    # 4) Optimize as usual
    v_opt = optim(v_init, fopt)

    # 5) combine fiexed and optimized parameters
    v_opt' = splice(v_opt, v_fix)

    # 6) user get a Named Tuple back
    return tt(v_opt')

end

It seems an common and not very difficult problem. Looking forward to any ideas!

2

The normal practice would be to capture fixed parameters with closures. e.g. optimize(p -> somefunction(p, a, b), pinit, ...) where a and b are fixed parameters.

(In NLopt, you can set upper and lower bounds on each variable, so another way to keep a parameter fixed is to set upper bound = lower bound.)

1 Like
3

Thanks! That’s my goal here (see fopt in my first attempt).

But my question is how I can construct such a closure automatically, given the Tuples p_init and p_keep_fix.