Fix parameter when passing to optimizer

Could you help me with a constructor?

FliggedNamedTuple(v::NamedTuple) = FliggedNamedTuple(ComponentArray(v), ComponentArray(v_with_values_replaced_by_bools))

struggling to covert the type preserving the structure

got it

	struct FliggedNamedTuple{T,Ax}
	function FliggedNamedTuple(v::NamedTuple)
	    cv = ComponentVector(v)
	    dof = length(cv)
	    axes = getaxes(cv)[1]
	    FliggedNamedTuple(cv, ComponentVector(fill(false,dof), axes))
1 Like

The way I like doing this is by composing the objective to be optimized with a function that completes the model. I think this is pretty much what @jonniedie suggested, and is as close as it gets to how you would formulate it on paper, so it cannot be too wrong :slight_smile:

1 Like

@lostella, thank for the links

what are parameters in your case?
how is the composition (\circ) defined?
Is m_original.layers global?

Exactly, in that specific example the m_original is an object in global scope, and it contains all parameters, some of which we want to fine-tune. So get_complete_model just takes the subset of parameters we want to optimize, and puts it together with the part of m_original that we want to keep.

In my example above, the model is the composition of multiple layers where we want to fine tune only the final ones: so it takes all layers but the last three from m_original and stacks the given layers on top of these. But this could work for anything that holds your parameters, including a ComponentArray from which you only want to optimize some components, or a regular Array from which you want to optimize some coefficients, or whatever.

The principle is simple: if f is a function of all parameters, then objective = f ∘ get_complete_model is a function of the subset of parameters that you want to optimize. The composition operator \circ is straight from base Julia, so the above is equivalent to objective = x -> f(get_complete_model(x)).

1 Like

ok, thanks for the clarification. Nice