How to create a Flux model from parameter vector?

New to Julia
1

Hi,
I have a parameter vector and shape array as below:

shape =[((2,5),2),((2,2),2)] #shape of layers : weights and bias

w = rand(18) #weights and bias together.

In this case elements of w are divided as below:

  • First 10 elements of w are weights of layer 1
  • Next 2 elements of w are bias of layer 1
  • First 4elements of w are weights of layer 2
  • Next 2 elements of w are bias of layer 2

What will be the best way to create a Flux model using like below using elements of w

Chain(Dense(5,2), Dense(2,2))

Thanks and Regards
Manu

2

I would first create the model using something like

model = Chain(map(x -> Dense(x[1]...), shape)...)

and then use loadparams!(model, theta),
where I would create theta as

theta = mapreduce(x -> [view(parameter_vector, x[1]...),view(parameter_vector, x[2])], vcat, shape)

Also, there is destructure and restructure commands, converting model to a vector of parameters and back, so I would take a look on those.

2 Likes
3

@Tomas_Pevny: I really appreciate your reply. Your solution saved my time… Amazing…
I looked on destructure

θ, re = destructure(model);

Then I can update model with new weight as

re(w)
1 Like
4

Yeah, destructure was written for exactly this use-case.

1 Like