# Flux train all parameters except those equal to 0

**URL:** <https://discourse.julialang.org/t/flux-train-all-parameters-except-those-equal-to-0/55665>\
**Category:** Machine Learning\
**Created:** [February 20, 2021, 8:48am UTC](https://discourse.julialang.org/t/flux-train-all-parameters-except-those-equal-to-0/55665 "2021-02-20T08:48:37Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![anon74562486](https://avatars.discourse-cdn.com/v4/letter/a/c68b51/32.png) [@anon74562486](https://discourse.julialang.org/u/anon74562486)\
**Post date:** [February 20, 2021, 8:48am UTC](https://discourse.julialang.org/t/flux-train-all-parameters-except-those-equal-to-0/55665/1 "2021-02-20T08:48:37Z")

</div>

I have a neural network model trained on the mnist dataset.  
If I add to the loss function a l1 regularization term I get many parameters near 0.  
But if I set manually those parameters to 0 the model performs worse and I have to train the model a little more to get the same performance.

How can I train all parameters except those equal to 0?  
Thank you.

---

<div class="post-metadata">

**Author:** ![anon74562486](https://avatars.discourse-cdn.com/v4/letter/a/c68b51/32.png) [@anon74562486](https://discourse.julialang.org/u/anon74562486)\
**Post date:** [April 2, 2021, 7:43pm UTC](https://discourse.julialang.org/t/flux-train-all-parameters-except-those-equal-to-0/55665/3 "2021-04-02T19:43:53Z")

</div>

```julia
p = params(model)
ott = ADAM()
for epoch in 1:epochs
    for (x, y) in data
        gr = gradient(p) do 
            loss(x, y)
        end

        for i in 1:length(p)
            gr[p[i]][p[i] .== 0f0] .= 0f0    
        end
        Flux.update!(ott, p, gr)
    end
end

```

this solution is inspired by this one: [How to freeze a single weight of a layer? - Usage - JuliaLang](https://discourse.julialang.org/t/how-to-freeze-a-single-weight-of-a-layer/58279/4)