# Does Knet support nested gradient for conv?

**URL:** <https://discourse.julialang.org/t/does-knet-support-nested-gradient-for-conv/30805>\
**Category:** Machine Learning\
**Created:** [November 6, 2019, 8:49pm UTC](https://discourse.julialang.org/t/does-knet-support-nested-gradient-for-conv/30805 "2019-11-06T20:49:17Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![zht](https://avatars.discourse-cdn.com/v4/letter/z/67e7ee/32.png) [@zht](https://discourse.julialang.org/u/zht)\
**Post date:** [November 6, 2019, 8:49pm UTC](https://discourse.julialang.org/t/does-knet-support-nested-gradient-for-conv/30805/1 "2019-11-06T20:49:17Z")

</div>

I need to take the grad of grad of an NN (this is needed for WGAN with grad penalty regularizations). I just found out that Flux (0.9 release) does not support it:

```julia
Nested AD not defined for conv

Stacktrace:
...

```

I am curious if Knet @denizyuret support nested AD for all network layers. I know this is supported in PyTorch.

---

<div class="post-metadata">

**Author:** ![denizyuret](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/denizyuret/32/568_2.png) [@denizyuret](https://discourse.julialang.org/u/denizyuret)\
**Post date:** [November 12, 2019, 6:30am UTC](https://discourse.julialang.org/t/does-knet-support-nested-gradient-for-conv/30805/2 "2019-11-12T06:30:15Z")

</div>

Unfortunately not. I recently implemented all the missing pieces for MLP nested gradients. I need to figure out the second gradients for conv to add support.