# How to implement a sparse decoder layer in Flux.jl?

**URL:** <https://discourse.julialang.org/t/how-to-implement-a-sparse-decoder-layer-in-flux-jl/123931>\
**Category:** New to Julia\
**Tags:** question, flux\
**Created:** [December 17, 2024, 6:27pm UTC](https://discourse.julialang.org/t/how-to-implement-a-sparse-decoder-layer-in-flux-jl/123931 "2024-12-17T18:27:59Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Flavio\_Barros](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/flavio_barros/32/45443_2.png) [@Flavio\_Barros](https://discourse.julialang.org/u/Flavio_Barros)\
**Post date:** [December 17, 2024, 6:27pm UTC](https://discourse.julialang.org/t/how-to-implement-a-sparse-decoder-layer-in-flux-jl/123931/1 "2024-12-17T18:27:59Z")

</div>

I’m new to Julia and Flux.jl and I’m trying to implement a Variational Autoencoder (VAE). My decoder has a single layer mapping to the output, but I need to apply different weights to each example.

Instead of a dense layer, I want to interact the weights with a matrix that indicates which weights should be non-zero for each example. Essentially, I’m looking for a way to implement a sparse decoder layer.

Is this possible in Flux? Could someone provide an example or point me to relevant documentation?

> # Define encoder
> 
> encoder(x) = Chain(  
> Dense(40, 20, relu),  
> Dense(20, 20)  
> )(x)
> 
> # Decoder 1 layer
> 
> decoder(z) = Dense(20, 7, sigmoid)

---

<div class="post-metadata">

**Author:** ![mcabbott](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mcabbott/32/6603_2.png) [@mcabbott](https://discourse.julialang.org/u/mcabbott)\
**Post date:** [December 17, 2024, 9:16pm UTC](https://discourse.julialang.org/t/how-to-implement-a-sparse-decoder-layer-in-flux-jl/123931/2 "2024-12-17T21:16:42Z")

</div>

> [@Flavio\_Barros](#):
>
> Instead of a dense layer, I want to interact the weights with a matrix that indicates which weights should be non-zero for each example.

Maybe you want this, although I’m not sure I’d recommend it:

```julia
julia> using SparseArrays

julia> Dense(sprand(5,3,0.2))
Dense(3 => 5) # 20 parameters <-- this is a lie

julia> ans.weight
5×3 SparseMatrixCSC{Float64, Int64} with 3 stored entries:
  ⋅ ⋅ ⋅ 
  ⋅ ⋅ 0.357995
  ⋅ ⋅ 0.353468
 0.552263 ⋅ ⋅ 
  ⋅ ⋅ ⋅ 

```

But if not, can you explain what you mean here? For instance, write out code which uses some weights and some “examples”, even if it’s not fast / Zygote-friendly.