# Custom layer in Lux

**URL:** <https://discourse.julialang.org/t/custom-layer-in-lux/117642>\
**Category:** Machine Learning\
**Tags:** lux\
**Created:** [July 30, 2024, 4:38pm UTC](https://discourse.julialang.org/t/custom-layer-in-lux/117642 "2024-07-30T16:38:53Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![rkube](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rkube/32/211198_2.png) [@rkube](https://discourse.julialang.org/u/rkube)\
**Post date:** [July 30, 2024, 4:38pm UTC](https://discourse.julialang.org/t/custom-layer-in-lux/117642/1 "2024-07-30T16:38:53Z")

</div>

Hi,  
I’m trying to implement a custom layer in Lux that is similar to a linear layer but needs to apply a user-defined mask to the weights. It needs to compute

```julia
h(x) = g(b + (W ⊙ M)(x))

```

where `g` is an activation function, `b` the bias vector, `W` the weight matrix and `M` the user-defined mask.

Some things that are unclear to me after reading the [documentation](https://lux.csail.mit.edu/stable/manual/interface):

1. The weight matrix in my layer is user-definable and may need to be updated. Should the matrix be defined in the struct definition of the layer or in the parameters?
2. My uses explicit function definitions as in the tutorial linked above, but there exists also the `@compact` macro. What exactly does this macro do?
3. The definition of [dense](https://github.com/LuxDL/Lux.jl/blob/59402fe726a88e20bd80f31cb408c02c8581b7af/src/layers/basic.jl#L363), calls `Lux._vec` and `Lux._getproperty`. How are these different from `vec` and `getproperty`?
4. What is the rationale for using `F1` and `F2` as type parameters in the tutorial?

```julia
struct Linear{F1, F2} <: LuxCore.AbstractExplicitLayer
    in_dims::Int
    out_dims::Int
    init_weight::F1
    init_bias::F2
end

```

---

<div class="post-metadata">

**Author:** ![avikpal](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/avikpal/32/6550_2.png) [@avikpal](https://discourse.julialang.org/u/avikpal)\
**Post date:** [July 31, 2024, 2:42am UTC](https://discourse.julialang.org/t/custom-layer-in-lux/117642/2 "2024-07-31T02:42:09Z")

</div>

> 1. The weight matrix in my layer is user-definable and may need to be updated. Should the matrix be defined in the struct definition of the layer or in the parameters?

In the parameters. Model structs should never contain mutable elements. See [Migrating from Flux to Lux | Lux.jl Docs](https://lux.csail.mit.edu/stable/manual/migrate_from_flux#migrate-from-flux).

> 1. My uses explicit function definitions as in the tutorial linked above, but there exists also the `@compact` macro. What exactly does this macro do?

See [Utilities | Lux.jl Docs](https://lux.csail.mit.edu/stable/api/Lux/utilities#Lux.@compact) for a detailed description. It essentially automatically writes all the boilerplate code needed for state handling and defining `initialparameters` / `initialstates`. See [this tutorial](https://lux.csail.mit.edu/stable/tutorials/beginner/3_SimpleRNN#Creating-a-Classifier) for a example showcasing both kinds of layers.

> 1. The definition of [dense](https://github.com/LuxDL/Lux.jl/blob/59402fe726a88e20bd80f31cb408c02c8581b7af/src/layers/basic.jl#L363), calls `Lux._vec` and `Lux._getproperty`. How are these different from `vec` and `getproperty`?

Mostly an implementation detail. `_getproperty` takes `Val` as input and returns nothing if no such field is present in the struct. `_vec` allows `nothing` as input.

> 1. What is the rationale for using `F1` and `F2` as type parameters in the tutorial?

Just to specialize on the functions