# How to do L2 regularization with new Flux and Zygote

**URL:** <https://discourse.julialang.org/t/how-to-do-l2-regularization-with-new-flux-and-zygote/31756>\
**Category:** Machine Learning\
**Tags:** flux\
**Created:** [December 2, 2019, 2:28pm UTC](https://discourse.julialang.org/t/how-to-do-l2-regularization-with-new-flux-and-zygote/31756 "2019-12-02T14:28:52Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![HenriDeh](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/henrideh/32/8316_2.png) [@HenriDeh](https://discourse.julialang.org/u/HenriDeh)\
**Post date:** [December 2, 2019, 2:28pm UTC](https://discourse.julialang.org/t/how-to-do-l2-regularization-with-new-flux-and-zygote/31756/1 "2019-12-02T14:28:52Z")

</div>

I think this MWE used to work. Now I don’t know how to compute a norm penalty because the example in Flux documentation [here](https://fluxml.ai/Flux.jl/stable/models/regularisation/) is still using the old Tracked stuff.

```julia
using Flux

m = Dense(5,1)

gs = Flux.gradient(Flux.params(m)) do
    sum(norm, Flux.params(m))
end

```

throws

```julia
ERROR: Mutating arrays is not supported
Stacktrace:
 [1] (::Zygote.var"#986#987")(::Nothing) at C:\Users\Henri\.julia\packages\Zygote\8dVxG\src\lib\array.jl:42
 [2] (::Zygote.var"#2607#back#988"{Zygote.var"#986#987"})(::Nothing) at C:\Users\Henri\.julia\packages\ZygoteRules\6nssF\src\adjoint.jl:49
 [3] push! at C:\Users\Henri\.julia\packages\Zygote\8dVxG\src\compiler\interface.jl:65 [inlined]
 [4] (::typeof(∂(push!)))(::Nothing) at C:\Users\Henri\.julia\packages\Zygote\8dVxG\src\compiler\interface2.jl:0
 [5] params! at C:\Users\Henri\.julia\packages\Flux\oX9Pi\src\functor.jl:42 [inlined]
 [6] (::typeof(∂(params!)))(::Nothing) at C:\Users\Henri\.julia\packages\Zygote\8dVxG\src\compiler\interface2.jl:0
 [7] params! at C:\Users\Henri\.julia\packages\Flux\oX9Pi\src\functor.jl:48 [inlined]
 [8] (::typeof(∂(params!)))(::Nothing) at C:\Users\Henri\.julia\packages\Zygote\8dVxG\src\compiler\interface2.jl:0
 [9] params! at C:\Users\Henri\.julia\packages\Flux\oX9Pi\src\functor.jl:48 [inlined]
 [10] (::typeof(∂(params!)))(::Nothing) at C:\Users\Henri\.julia\packages\Zygote\8dVxG\src\compiler\interface2.jl:0
 [11] params! at C:\Users\Henri\.julia\packages\Flux\oX9Pi\src\functor.jl:48 [inlined]
 [12] (::typeof(∂(params!)))(::Nothing) at C:\Users\Henri\.julia\packages\Zygote\8dVxG\src\compiler\interface2.jl:0
 [13] params! at C:\Users\Henri\.julia\packages\Flux\oX9Pi\src\functor.jl:45 [inlined]
 [14] (::typeof(∂(params!)))(::Nothing) at C:\Users\Henri\.julia\packages\Zygote\8dVxG\src\compiler\interface2.jl:0
 [15] params at C:\Users\Henri\.julia\packages\Flux\oX9Pi\src\functor.jl:54 [inlined]
 [16] (::typeof(∂(params)))(::NamedTuple{(:order, :params),Tuple{Array{Array{Float64,N} where N,1},Nothing}}) at C:\Users\Henri\.julia\packages\Zygote\8dVxG\src\compiler\interface2.jl:0
 [17] #48 at .\none:2 [inlined]
 [18] (::typeof(∂(#48)))(::Float64) at C:\Users\Henri\.julia\packages\Zygote\8dVxG\src\compiler\interface2.jl:0
 [19] (::Zygote.var"#38#39"{Zygote.Params,Zygote.Context,typeof(∂(#48))})(::Float64) at C:\Users\Henri\.julia\packages\Zygote\8dVxG\src\compiler\interface.jl:101
 [20] gradient(::Function, ::Zygote.Params) at C:\Users\Henri\.julia\packages\Zygote\8dVxG\src\compiler\interface.jl:47
 [21] top-level scope at none:0

```

---

<div class="post-metadata">

**Author:** ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)\
**Post date:** [December 2, 2019, 10:57pm UTC](https://discourse.julialang.org/t/how-to-do-l2-regularization-with-new-flux-and-zygote/31756/2 "2019-12-02T22:57:35Z")

</div>

Sum with function argument is also known to cause bugs with zygote  
[https://github.com/FluxML/Zygote.jl/issues/231](https://github.com/FluxML/Zygote.jl/issues/231)

---

<div class="post-metadata">

**Author:** ![chengchingwen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chengchingwen/32/8390_2.png) [@chengchingwen](https://discourse.julialang.org/u/chengchingwen)\
**Post date:** [December 29, 2019, 3:39pm UTC](https://discourse.julialang.org/t/how-to-do-l2-regularization-with-new-flux-and-zygote/31756/3 "2019-12-29T15:39:31Z")

</div>

Better use `Optimiser(WeightDecay(lambda), opt)` for l2 regularization.