# Writing complex Flux Models

**URL:** <https://discourse.julialang.org/t/writing-complex-flux-models/40439>\
**Category:** Machine Learning\
**Created:** [May 29, 2020, 8:06pm UTC](https://discourse.julialang.org/t/writing-complex-flux-models/40439 "2020-05-29T20:06:42Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![irhum](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/irhum/32/25000_2.png) [@irhum](https://discourse.julialang.org/u/irhum)\
**Post date:** [May 29, 2020, 8:06pm UTC](https://discourse.julialang.org/t/writing-complex-flux-models/40439/1 "2020-05-29T20:06:42Z")

</div>

Flux’s typical go to when writing models is Chains, and for most feedforward models (with the occasional skip connection), with one input, it works quite beautifully.

Problem is, when defining models as functions, we lose the ability to get their params directly, or move them to the GPU. Even as the example given in the Flux docs

```julia
function linear(in, out)
  W = randn(out, in)
  b = randn(out)
  x -> W * x .+ b
end

```

results in an empty parameter list

```julia
model = linear(3, 2)
params(model) #Params([])

```

would it be possible to extend the Flux.@functor machinery so that models defined in a format like this, for instance

```julia
function custommodel() # defines model parameters
    A = Dense(20, 40)
    B = Dense(40, 60)
    C = Dense(60, 20)
    return function(L, M, N, O) # defines the forward pass
        H = A(L)
        H = H * M .+ B(N) * O
        H = C(H) + L
    end

```

or any other forward pass, with any number of inputs can be written, while still returning having methods like `params()`, `gpu()` and so on be available? The output of custommodel() is effectively an anonymous type as well, for instance,

```julia
mymodel = custommodel()
mymodel.B # Dense(40, 60)

```

since Flux.@functor already works on user-defined types to extend params() and gpu() to them, would it be possible to extend to these anonymous types as well? It’d be much cleaner to write more complex models if this were to be the case

---

<div class="post-metadata">

**Author:** ![findmyway](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/findmyway/32/4946_2.png) [@findmyway](https://discourse.julialang.org/u/findmyway)\
**Post date:** [June 2, 2020, 6:26am UTC](https://discourse.julialang.org/t/writing-complex-flux-models/40439/2 "2020-06-02T06:26:34Z")

</div>

So why not just define your `custommodel` as `struct` here?

---

<div class="post-metadata">

**Author:** ![irhum](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/irhum/32/25000_2.png) [@irhum](https://discourse.julialang.org/u/irhum)\
**Post date:** [June 2, 2020, 5:08pm UTC](https://discourse.julialang.org/t/writing-complex-flux-models/40439/3 "2020-06-02T17:08:47Z")

</div>

It’s definitely a no-brainer to do that, thing is structs can’t be redefined the way Chains can, which can make iterating on a model, especially when making changes in network structure somewhat tiresome. It’s hardly a breaking issue admittedly, but it would be a nice QOL improvement

---

<div class="post-metadata">

**Author:** ![DrChainsaw](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/drchainsaw/32/8497_2.png) [@DrChainsaw](https://discourse.julialang.org/u/DrChainsaw)\
**Post date:** [June 2, 2020, 6:57pm UTC](https://discourse.julialang.org/t/writing-complex-flux-models/40439/4 "2020-06-02T18:57:08Z")

</div>

I think I’ve seen somewhere that closures being anonymous structs is an implementation detail which it is not recommended to depend on. Of course, if it is a single purpose project then by all means, just go ahead and implement that functor in there and use it while it works.

```julia
# Too lazy to load Flux, I think something like this might work except closurefunctor shall be Flux.functor
julia> function test(a, b)
       return function(x)
          return a .+ b .* x
       end
       end
test (generic function with 2 methods)

julia> tt = test(1, [2,3]);

julia> fieldnames(typeof(tt))
(:a, :b)

julia> closurefunctor(x) = map(fn -> getfield(x, fn), fieldnames(typeof(x))), test
closurefunctor (generic function with 1 method)

julia> p,re = closurefunctor(tt);

julia> p
(1, [2, 3])

julia> re(4, [1,2,3])
#21 (generic function with 1 method)

julia> ttt = re(4, [1,2,3])
#21 (generic function with 1 method)

julia> ttt(3)
3-element Array{Int64,1}:
  7
 10
 13

```

**Shameless plug** : This package supports a static computation graph format designed to be created and manipulated programatically: [GitHub - DrChainsaw/NaiveNASflux.jl: Your local Flux surgeon](https://github.com/DrChainsaw/NaiveNASflux.jl)

---

<div class="post-metadata">

**Author:** ![dunefox](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dunefox/32/12149_2.png) [@dunefox](https://discourse.julialang.org/u/dunefox)\
**Post date:** [June 5, 2020, 7:14pm UTC](https://discourse.julialang.org/t/writing-complex-flux-models/40439/5 "2020-06-05T19:14:05Z")

</div>

> [@irhum](#):
>
> It’s definitely a no-brainer to do that, thing is structs can’t be redefined the way Chains can

Doesn’t putting the model struct into a module solve that problem? I think you can redefine it then.