# Calling Flux.params() inside gradient changes output?

**URL:** <https://discourse.julialang.org/t/calling-flux-params-inside-gradient-changes-output/68829>\
**Category:** Machine Learning\
**Tags:** flux, zygote\
**Created:** [September 27, 2021, 7:18pm UTC](https://discourse.julialang.org/t/calling-flux-params-inside-gradient-changes-output/68829 "2021-09-27T19:18:22Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![AlexLewandowski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/alexlewandowski/32/18856_2.png) [@AlexLewandowski](https://discourse.julialang.org/u/AlexLewandowski)\
**Post date:** [September 27, 2021, 7:18pm UTC](https://discourse.julialang.org/t/calling-flux-params-inside-gradient-changes-output/68829/1 "2021-09-27T19:18:22Z")

</div>

Could anyone help me understand why calling `Flux.params()` inside a gradient, changes the gradient? Below is a minimal example where, in `get_grad_2`, I instantiate a variable `prowling_variable = Flux.params(m)`. The gradients returned in `g1` are all `0.0`, but `g2` outputs all `1.0`. It is as if the loss has `sum(prowling_variable)` as a regularizer. You can see that g3.grads == g2.grads, suggesting that calling Flux.params is equivalent to summing the parameters and adding it to the loss.

I know you can avoid this issue by storing the variables in a struct and not calling `Flux.params` in `gradient`. But this seems like it could be a bug.

```julia
using Flux, BenchmarkTools

m = Chain(Dense(100, 50, relu), Dense(50, 2), softmax);

opt = Descent(0.01);

data, labels = rand(Float32, 100, 100), zeros(Float32, 2, 100);

loss(m, x, y) = sum(Flux.crossentropy(m(x), y));

function get_grad(m, data, labels)
            gs = gradient(Flux.params(m)) do
              l = loss(m, data, labels)
           end
         end

function get_grad_2(m, data, labels)
            gs = gradient(Flux.params(m)) do
              prowling_variable = Flux.params(m)
              l = loss(m, data, labels)
           end
         end

function get_grad_3(m, data, labels)
            gs = gradient(Flux.params(m)) do
              l = loss(m, data, labels) + sum([sum(p) for p in Flux.params(m)])
           end
         end

g1 = get_grad(m, data, labels)
g2 = get_grad_2(m, data, labels)
g3 = get_grad_3(m, data, labels)

println(g1.grads == g2.grads)
println(g2.grads == g3.grads)

```

---

<div class="post-metadata">

**Author:** ![Christian\_Rorvik](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/christian_rorvik/32/965_2.png) [@Christian\_Rorvik](https://discourse.julialang.org/u/Christian_Rorvik)\
**Post date:** [September 28, 2021, 5:46am UTC](https://discourse.julialang.org/t/calling-flux-params-inside-gradient-changes-output/68829/2 "2021-09-28T05:46:45Z")

</div>

I don’t have an answer, but I’ve been confused by the same behaviour in my own code and had to work around it.

---

<div class="post-metadata">

**Author:** ![ToucheSir](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/touchesir/32/14411_2.png) [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)\
**Post date:** [September 28, 2021, 6:32pm UTC](https://discourse.julialang.org/t/calling-flux-params-inside-gradient-changes-output/68829/3 "2021-09-28T18:32:16Z")

</div>

This is almost certainly a bug. I dropped an MWE on the most likely [PR](https://github.com/FluxML/Zygote.jl/pull/992) culprit, so feel free to follow/comment on there.