# How to initialize a subset of params of a NN

**URL:** <https://discourse.julialang.org/t/how-to-initialize-a-subset-of-params-of-a-nn/33423>\
**Category:** New to Julia\
**Tags:** flux\
**Created:** [January 15, 2020, 11:02pm UTC](https://discourse.julialang.org/t/how-to-initialize-a-subset-of-params-of-a-nn/33423 "2020-01-15T23:02:27Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![shrirams](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/shrirams/32/12323_2.png) [@shrirams](https://discourse.julialang.org/u/shrirams)\
**Post date:** [January 15, 2020, 11:02pm UTC](https://discourse.julialang.org/t/how-to-initialize-a-subset-of-params-of-a-nn/33423/1 "2020-01-15T23:02:27Z")

</div>

Suppose I have a trained neural net  
`nn1 = Flux.Chain(Dense(90, 60, relu), Dense(60, 30, relu), Dense(30,1)).`

I am aware that I can access the weights corresponding to the first 2 layers by params(nn1[1]), params(nn1[2]).

However, I then set up  
`nn2 = Flux.Chain(Dense(90, 60, relu), Dense(60, 30, relu), Dense(30,2))`.

Note that nn1 and nn2 have the same structure in first 2 layers.

Can I initialize the weights of the first two layers with

```julia
params(nn2[1]) = params(nn1[1])
 params(nn2[2]) = params(nn1[2])

```

---

<div class="post-metadata">

**Author:** ![Skoffer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/skoffer/32/378_2.png) [@Skoffer](https://discourse.julialang.org/u/Skoffer)\
**Post date:** [January 16, 2020, 7:32am UTC](https://discourse.julialang.org/t/how-to-initialize-a-subset-of-params-of-a-nn/33423/2 "2020-01-16T07:32:49Z")

</div>

I wonder what is the most idiomatic answer to this question, but you may go to low level.

Since `Dense` is just an ordinary immutable structure which contains data, you can change it’s weights directly.

```julia
nn2[1].W .= nn1[1].W
nn2[1].b .= nn1[1].b

```

or you can wrap it (unsafe) function

```julia
function Base.:setindex!(c::Chain, d::Dense, i)
    c.layers[i].W .= d.W
    c.layers[i].b .= d.b
end

nn2[1] = nn1[1]
nn2[2] = nn1[2]

```

But I think it’s better to construct `nn2` from `nn1` directly

```julia
nn2 = Flux.Chain(deepcopy(nn1[1:2])..., Dense(30, 2))

```

---

<div class="post-metadata">

**Author:** ![Skoffer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/skoffer/32/378_2.png) [@Skoffer](https://discourse.julialang.org/u/Skoffer)\
**Post date:** [January 16, 2020, 8:22am UTC](https://discourse.julialang.org/t/how-to-initialize-a-subset-of-params-of-a-nn/33423/3 "2020-01-16T08:22:21Z")

</div>

Also you may use `loadparams!` function.

```julia
Flux.loadparams!(nn2[1:2], params(nn1[1:2]))

```

It is more generic approach, since it doesn’t rely on internal structure of `Dense`. If you want, you may add some sugar

```julia
Base.:setindex!(c::Chain, x, i) = Flux.loadparams!(c[i], params(x))
nn2[1:2] = nn1[1:2]

```