I am hoping to implement a custom “layer” soon.
- I think the requirements for this are
a) it accepts input from previous layer, and produces outputs,
b) Flux.params() can find its parameters,
c) gpu() can find its parameters to transfer to the gpu.
Is this sort-of correct?
- How to cause b),c) to happen?
This flux-custom-model-feedback-of-the-output-layer-to-the-input-layer discussion indicates that overriding
Flux.@functor mytypeare involved.
However, what is the difference between Flux.trainable and Flux.@functor?
The documentation seems to suggest that these have the same purpose.
The first way of achieving this is through overloading the
Another way of achieving this is through the
Actually, I think that the should be different. A model might need some extra arrays, that are not directly parameters for AD, and gpu() should move these extra arrays to the gpu. (Just a guess, I cannot give an example).