# Flux & Float 64 + model deconstruction?
**URL:** https://discourse.julialang.org/t/flux-float-64-model-deconstruction/32048
**Category:** Machine Learning
**Created:** [December 9, 2019, 2:06pm UTC](https://discourse.julialang.org/t/flux-float-64-model-deconstruction/32048 "2019-12-09T14:06:03Z")
**Posts on this page:** 11
**Page:** 1
### Author:  [@BLI](https://discourse.julialang.org/u/BLI)
#### Post date: [December 9, 2019, 2:06pm UTC](https://discourse.julialang.org/t/flux-float-64-model-deconstruction/32048/1 "2019-12-09T14:06:03Z")
I’m trying to run my past Flux networks, and have a couple of questions.
First, I figured out that with model `m` and data `X`, I now find predictions by `m(X)` instead of the old `Tracker.data(m(X))` — great, I like that.
Next to my three questions.
1. How can I tell the network (or an individual layer) that I want the parameters (weights `W`, biases `b`) to be `Float64` — if I want to (I’m running the code on the CPU).
2. How can I “deconstruct” the network and pick out weights and parameters for each layer?
3. How can I initiate each layer with my choice of parameters? [e.g., those from another, “deconstructed” model…]
---
### Author:  [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)
#### Post date: [December 9, 2019, 4:24pm UTC](https://discourse.julialang.org/t/flux-float-64-model-deconstruction/32048/2 "2019-12-09T16:24:55Z")
I think this exact problem is substantially easier to solve with Flux 0.10, so I’d upgrade to handle this.
---
### Author:  [@BLI](https://discourse.julialang.org/u/BLI)
#### Post date: [December 9, 2019, 5:23pm UTC](https://discourse.julialang.org/t/flux-float-64-model-deconstruction/32048/3 "2019-12-09T17:23:15Z")
I should have been more precise: I do have Flux v.0.10 installed. I figured out how to deconstruct a dense/FNN with the past version, but had problems with selecting parameter types then.
So I’m curious about how to deconstruct an FNN (Feedforward = Dense) network under v. 0.10 + specify the parameter type + populate the parameters with values of my own choosing.
Is this documented somewhere?
---
### Author:  [@BLI](https://discourse.julialang.org/u/BLI)
#### Post date: [December 9, 2019, 6:03pm UTC](https://discourse.julialang.org/t/flux-float-64-model-deconstruction/32048/4 "2019-12-09T18:03:58Z")
OK… I see that I can deconstruct the model `m` by taking out each layer as `m[1]`, `m[2]`, etc.
Next, I can find the parameters as `m[1].W` and `m[1].b`.
**However** , I can not _change_ the parameters by, say, `m[1].W = rand(Float64,5,2)`.
QUESTION: How can I change the values of the parameters?
---
### Author:  [@simeonschaub](https://discourse.julialang.org/u/simeonschaub)
#### Post date: [December 9, 2019, 6:14pm UTC](https://discourse.julialang.org/t/flux-float-64-model-deconstruction/32048/5 "2019-12-09T18:14:31Z")
You can do `m[1].W .= rand(Float64,5,2)`. You’re trying to bind the `W` parameter of `m[1]` to a different array, which doesn’t work if `m[1]` is immutable. `.=` writes the values into the already existing matrix `W` which does work because arrays are always mutable.
---
### Author:  [@BLI](https://discourse.julialang.org/u/BLI)
#### Post date: [December 9, 2019, 6:41pm UTC](https://discourse.julialang.org/t/flux-float-64-model-deconstruction/32048/6 "2019-12-09T18:41:39Z")
Ah. I should have guessed that. I assume I can also change type, if the original type is Float32, I can change it to Float64?
---
### Author:  [@dellison](https://discourse.julialang.org/u/dellison)
#### Post date: [December 9, 2019, 6:57pm UTC](https://discourse.julialang.org/t/flux-float-64-model-deconstruction/32048/7 "2019-12-09T18:57:49Z")
I don’t have a Julia installation at work (so I can’t test this at the moment), but with Flux 0.10.0 you should be able to do #1 on your list this way:
```julia
model = fmap(f64, model)
```
Flux’s `fmap` function lets you operate on all the model’s parameters at once, and the `f64` function changes things to have a `Float64` type. (In Flux version 0.9.0 and earlier, you can use `mapleaves` instead of `fmap` for this purpose.)
You can look at the constructors for the particular layers you’re interested in, too- a lot of them, at least, let you pass in an array to initialize the parameters. For example, `Dense` has the `initW` and `initb`keyword arguments that you can use here ([see here](https://github.com/FluxML/Flux.jl/blob/f46b5243dbc762081ccfbe991690750b307e2fc2/src/layers/basic.jl#L93)). Similarly, `Conv` has positional arguments for this ([here](https://github.com/FluxML/Flux.jl/blob/f46b5243dbc762081ccfbe991690750b307e2fc2/src/layers/conv.jl#L35)).
These aren’t really obvious to me from the documentation, so I think that could perhaps use a little improvement. But in the meantime, short of reading the source code, you can use `methods(Dense)` for example at the REPL to see what constructors are available to you.
I hope this helps! 🙂
---
### Author:  [@BLI](https://discourse.julialang.org/u/BLI)
#### Post date: [December 9, 2019, 8:53pm UTC](https://discourse.julialang.org/t/flux-float-64-model-deconstruction/32048/8 "2019-12-09T20:53:24Z")
Nope. Doesn’t work:
…and why not `map(Float64,mod)`?
Also:
So – seems like one cannot change the type after the network has been created…
Then, how can I define the type upon creation of the network?
---
### Author:  [@dellison](https://discourse.julialang.org/u/dellison)
#### Post date: [December 9, 2019, 9:04pm UTC](https://discourse.julialang.org/t/flux-float-64-model-deconstruction/32048/9 "2019-12-09T21:04:54Z")
I believe that `fmap` returns a copy of the model with the updated parameters, instead of doing it in-place. If you do `mod = fmap(f64, mod)` are all of `mod`’s parameters still `Float32`?
Edit: this seems to work for me, on Flux version 0.10.0:
```julia
julia> using Flux
[ Info: CUDAdrv.jl failed to initialize, GPU functionality unavailable (set JULIA_CUDA_SILENT or JULIA_CUDA_VERBOSE to silence or expand this message)
julia> m = Chain(Dense(4, 4, relu), Dense(4, 4), softmax)
Chain(Dense(4, 4, relu), Dense(4, 4), softmax)
julia> m[1].W
4×4 Array{Float32,2}:
0.823812 -0.593816 -0.799553 -0.570861
-0.242723 -0.529529 0.012944 0.740021
0.00234743 0.542825 -0.627849 -0.746003
0.640687 0.0562518 0.183272 0.300056
julia> m = fmap(f64, m)
Chain(Dense(4, 4, relu), Dense(4, 4), softmax)
julia> m[1].W
4×4 Array{Float64,2}:
0.823812 -0.593816 -0.799553 -0.570861
-0.242723 -0.529529 0.012944 0.740021
0.00234743 0.542825 -0.627849 -0.746003
0.640687 0.0562518 0.183272 0.300056
```
And, in fact, I’ve just learned that you can skip the `fmap` altogether and just call `f64` on the model directly. 🙂
```julia
julia> m = Chain(Dense(4, 4, relu), softmax) |> f64
Chain(Dense(4, 4, relu), softmax)
julia> m[1].W
4×4 Array{Float64,2}:
-0.378957 0.74928 0.776147 0.804017
-0.298902 -0.494922 -0.519403 -0.668515
0.0772961 -0.219601 0.776379 -0.067537
0.64462 -0.259356 -0.697054 -0.218149
```
---
### Author:  [@BLI](https://discourse.julialang.org/u/BLI)
#### Post date: [December 10, 2019, 8:45am UTC](https://discourse.julialang.org/t/flux-float-64-model-deconstruction/32048/10 "2019-12-10T08:45:00Z")
Thanks – first one worked for me, too (`m = fmap(f64,m)`). I assume the other also works – i.e., when instantiating the network! I’ll check when I get my laptop up and going.
Additional question… the `fmap` method is introduced in the documentation as `fmap(cu,m)` where `m` is the model, and I understand `cu` is meant to move the computations to a GPU??
Can this be used to run Flux on the NVIDIA 1050 GPU of my laptop – just to experiment with things?
---
### Author:  [@dellison](https://discourse.julialang.org/u/dellison)
#### Post date: [December 10, 2019, 6:53pm UTC](https://discourse.julialang.org/t/flux-float-64-model-deconstruction/32048/11 "2019-12-10T18:53:30Z")
Pretty much, yep! I don’t have a whole lot of experience with Flux on the GPU, but I think that it should work as long as you have CUDA/CUDNN installed. The `gpu` function is for moving models/data to the GPU: you can see [here](https://github.com/FluxML/Flux.jl/blob/08804a06d242224c12e1ca465cde5fadf4915b8e/src/functor.jl#L76) that it really just calls `cu` if there’s a GPU available. And, like with `f64`, you should be able to skip the `fmap` if you want to:
```julia
julia> model = Chain(Dense(10, 3), softmax) |> gpu
```
You’ve probably seen it, but there’s some documentation [here](https://fluxml.ai/Flux.jl/stable/gpu/).