# Fix parameter when passing to optimizer

**URL:** <https://discourse.julialang.org/t/fix-parameter-when-passing-to-optimizer/61102>\
**Category:** General Usage\
**Tags:** data\_structures\
**Created:** [May 13, 2021, 6:56pm UTC](https://discourse.julialang.org/t/fix-parameter-when-passing-to-optimizer/61102 "2021-05-13T18:56:23Z")\
**Posts on this page:** 7\
**Page:** 2

<div class="post-metadata">

**Author:** ![misha\_mikhasenko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/misha_mikhasenko/32/5060_2.png) [@misha\_mikhasenko](https://discourse.julialang.org/u/misha_mikhasenko)\
**Post date:** [May 15, 2021, 8:12pm UTC](https://discourse.julialang.org/t/fix-parameter-when-passing-to-optimizer/61102/21 "2021-05-15T20:12:09Z")

</div>

Could you help me with a constructor?

```julia
FliggedNamedTuple(v::NamedTuple) = FliggedNamedTuple(ComponentArray(v), ComponentArray(v_with_values_replaced_by_bools))

```

struggling to covert the type preserving the structure

EDIT:  
got it

```julia
begin
	struct FliggedNamedTuple{T,Ax}
	   parameters::ComponentVector{T,Vector{T},Ax}
	   flags::ComponentVector{Bool,Vector{Bool},Ax}
	end
	# 
	function FliggedNamedTuple(v::NamedTuple)
	    cv = ComponentVector(v)
	    dof = length(cv)
	    axes = getaxes(cv)[1]
	    FliggedNamedTuple(cv, ComponentVector(fill(false,dof), axes))
	end
end

```

---

<div class="post-metadata">

**Author:** ![lostella](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lostella/32/356_2.png) [@lostella](https://discourse.julialang.org/u/lostella)\
**Post date:** [May 17, 2021, 6:12pm UTC](https://discourse.julialang.org/t/fix-parameter-when-passing-to-optimizer/61102/22 "2021-05-17T18:12:36Z")

</div>

The way I like doing this is by [composing the objective to be optimized](https://github.com/lostella/ProtoGrad.jl/blob/6edc33e7198d8505dde7fae0cf477f68f45fe10e/examples/03_mnist_conv_finetune.jl#L36) with a [function that completes the model](https://github.com/lostella/ProtoGrad.jl/blob/6edc33e7198d8505dde7fae0cf477f68f45fe10e/examples/03_mnist_conv_finetune.jl#L20-L25). I think this is pretty much what @jonniedie suggested, and is as close as it gets to how you would formulate it on paper, so it cannot be too wrong 🙂

---

<div class="post-metadata">

**Author:** ![misha\_mikhasenko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/misha_mikhasenko/32/5060_2.png) [@misha\_mikhasenko](https://discourse.julialang.org/u/misha_mikhasenko)\
**Post date:** [May 18, 2021, 8:13am UTC](https://discourse.julialang.org/t/fix-parameter-when-passing-to-optimizer/61102/23 "2021-05-18T08:13:05Z")

</div>

@lostella, thank for the links

what are parameters in your case?  
how is the composition (\circ) defined?  
Is `m_original.layers` global?

---

<div class="post-metadata">

**Author:** ![lostella](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lostella/32/356_2.png) [@lostella](https://discourse.julialang.org/u/lostella)\
**Post date:** [May 18, 2021, 10:45am UTC](https://discourse.julialang.org/t/fix-parameter-when-passing-to-optimizer/61102/24 "2021-05-18T10:45:15Z")

</div>

Exactly, in that specific example the `m_original` is an object in global scope, and it contains all parameters, some of which we want to fine-tune. So `get_complete_model` just takes the subset of parameters we want to optimize, and puts it together with the part of `m_original` that we want to keep.

In my example above, the model is the composition of multiple layers where we want to fine tune only the final ones: so it takes all layers but the last three from `m_original` and stacks the given layers on top of these. But this could work for anything that holds your parameters, including a `ComponentArray` from which you only want to optimize some components, or a regular `Array` from which you want to optimize some coefficients, or whatever.

The principle is simple: if `f` is a function of _all_ parameters, then `objective = f ∘ get_complete_model` is a function of the subset of parameters that you want to optimize. The composition operator `\circ` is straight from base Julia, so the above is equivalent to `objective = x -> f(get_complete_model(x))`.

---

<div class="post-metadata">

**Author:** ![misha\_mikhasenko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/misha_mikhasenko/32/5060_2.png) [@misha\_mikhasenko](https://discourse.julialang.org/u/misha_mikhasenko)\
**Post date:** [May 18, 2021, 11:35am UTC](https://discourse.julialang.org/t/fix-parameter-when-passing-to-optimizer/61102/25 "2021-05-18T11:35:53Z")

</div>

ok, thanks for the clarification. Nice

---

<div class="post-metadata">

**Author:** ![roflmaostc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/roflmaostc/32/30123_2.png) [@roflmaostc](https://discourse.julialang.org/u/roflmaostc)\
**Post date:** [May 17, 2022, 4:09pm UTC](https://discourse.julialang.org/t/fix-parameter-when-passing-to-optimizer/61102/26 "2022-05-17T16:09:49Z")

</div>

Hi all!

I just stumbled over this post since we’ve got a similar problem where we want to fix certain parameters.

The main piece of code is:

```julia
function forward_with_fixed_params(forward, params)
           forward_fixed(x) = forward((; params..., x...))
           return forward_fixed
end

```

Is there anything wrong with that?

```julia
julia> forward((a, b, c)) = a+b+c^C

julia> forward(x) = x.a+x.b+x.c
forward (generic function with 1 method)

julia> forward((;a=0, b=10, c=10))
20

julia> fwd_n = forward_with_fixed_params(forward, (;a=10, b=10, c=10))
(::var"#forward_fixed#1"{typeof(forward), NamedTuple{(:a, :b, :c), Tuple{Int64, Int64, Int64}}}) (generic function with 1 method)

julia> fwd_n((;a=100))
120

julia> @code_warntype fwd_n((;a=100))
MethodInstance for (::var"#forward_fixed#1"{typeof(forward), NamedTuple{(:a, :b, :c), Tuple{Int64, Int64, Int64}}})(::NamedTuple{(:a,), Tuple{Int64}})
  from (::var"#forward_fixed#1")(x) in Main at REPL[1]:2
Arguments
  #self#::var"#forward_fixed#1"{typeof(forward), NamedTuple{(:a, :b, :c), Tuple{Int64, Int64, Int64}}}
  x::NamedTuple{(:a,), Tuple{Int64}}
Body::Int64
1 ─ %1 = Core.getfield(#self#, :forward)::Core.Const(forward)
│ %2 = Base.NamedTuple()::Core.Const(NamedTuple())
│ %3 = Core.getfield(#self#, :params)::NamedTuple{(:a, :b, :c), Tuple{Int64, Int64, Int64}}
│ %4 = Base.merge(%2, %3)::NamedTuple{(:a, :b, :c), Tuple{Int64, Int64, Int64}}
│ %5 = Base.merge(%4, x)::NamedTuple{(:a, :b, :c), Tuple{Int64, Int64, Int64}}
│ %6 = (%1)(%5)::Int64
└── return %6

julia> @time fwd_n((;a=100))
  0.000004 seconds
120

```

---

<div class="post-metadata">

**Author:** ![misha\_mikhasenko](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/misha_mikhasenko/32/5060_2.png) [@misha\_mikhasenko](https://discourse.julialang.org/u/misha_mikhasenko)\
**Post date:** [April 12, 2025, 3:16pm UTC](https://discourse.julialang.org/t/fix-parameter-when-passing-to-optimizer/61102/27 "2025-04-12T15:16:23Z")

</div>

Yes, I’m happy with this solution, that is what I do when dealing with NamedTuples.  
For `ComponentArrays` an analogous merging method did not exist.

Just added PR to be able to update deeply nested structures.

> <https://github.com/SciML/ComponentArrays.jl/issues/303>
>
> \## Motivation
> 
> In fitting or optimization workflows, it’s common to work with a …structured \`ComponentArray\` template and update only a subset of its parameters during each iteration. While manual assignment is always possible using field names or indices, it’s often verbose and error-prone.
> 
> I'd like to have a utility function that enables updating parameters by name using existing axis metadata, without reconstructing the structure or altering types.
> 
> \## Proposed Functionality
> Introduce \`update\_component\_array(default, update)\` that:
> \- Updates a \`ComponentArray\` default with values from another \`ComponentArray\` update
> \- Modifies only matching entries (partial updates supported)
> \- Preserves structure, type, and axis information of the original array
> 
> \## Example Usage
> 
> \`\`\`julia
> default = ComponentArray(sig = (mu = 1.0, sigma = 2.0), bg = 3.0)
> update = ComponentArray(sig = (mu = 1.1,), bg = 3.3)
> 
> result = update\_component\_array(default, update)
> 
> result.sig.mu # 1.1
> result.sig.sigma # 2.0
> result.bg # 3.3
> \`\`\`
> 
> \## Btw
> Originally, I tried to implement \`deep\_merge\`, but it's really hard to make it type stable. I'd be happy to have the \`update\` solution

[Previous page](https://discourse.julialang.org/t/fix-parameter-when-passing-to-optimizer/61102.md?page=1)