# Flux gradient clipping

**URL:** https://discourse.julialang.org/t/flux-gradient-clipping/18544
**Category:** Machine Learning
**Tags:** question
**Created:** [December 11, 2018, 4:33am UTC](https://discourse.julialang.org/t/flux-gradient-clipping/18544 "2018-12-11T04:33:30Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![Fart](https://avatars.discourse-cdn.com/v4/letter/f/41988e/32.png) [@Fart](https://discourse.julialang.org/u/Fart)
#### Post date: [December 11, 2018, 4:33am UTC](https://discourse.julialang.org/t/flux-gradient-clipping/18544/1 "2018-12-11T04:33:30Z")

</div>

Does anyone have a working implementation of gradient clipping in Flux? I’ve tried directly modifying the param.grad field but get `LoadError: type TrackedArray is immutable`

---

<div class="post-metadata">

### Author: ![Tomas\_Pevny](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tomas_pevny/32/25466_2.png) [@Tomas\_Pevny](https://discourse.julialang.org/u/Tomas_Pevny)
#### Post date: [December 11, 2018, 6:05am UTC](https://discourse.julialang.org/t/flux-gradient-clipping/18544/2 "2018-12-11T06:05:22Z")

</div>

Can’t you define a spacial layer, whose main goal would be just to clip the gradient? Something along the following line.

```julia
clip(x) = x
∇clip(Δ) = min.(max.(Δ, lower_bound), upperbound)

Flux.Tracker.@grad function clip(x)
        clip(Flux.data(x)), Δ -> (∇clip(Δ),)
    end

```

I think that if you register this, it should achieve your goal.

---

<div class="post-metadata">

### Author: ![MikeInnes](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mikeinnes/32/3656_2.png) [@MikeInnes](https://discourse.julialang.org/u/MikeInnes)
#### Post date: [December 11, 2018, 7:21pm UTC](https://discourse.julialang.org/t/flux-gradient-clipping/18544/3 "2018-12-11T19:21:04Z")

</div>

Flux provides `hook` for this. `hook(x, clip)` will apply `clip` to the gradient of `x`, and within that you can do whatever you want.