# ANN: JuliaSmoothOptimisers just got smoother - reverse-mode AD support

**URL:** <https://discourse.julialang.org/t/ann-juliasmoothoptimisers-just-got-smoother-reverse-mode-ad-support/58966>\
**Category:** Package Announcements\
**Tags:** optimization\
**Created:** [April 10, 2021, 6:17am UTC](https://discourse.julialang.org/t/ann-juliasmoothoptimisers-just-got-smoother-reverse-mode-ad-support/58966 "2021-04-10T06:17:43Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![mohamed82008](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mohamed82008/32/18171_2.png) [@mohamed82008](https://discourse.julialang.org/u/mohamed82008)\
**Post date:** [April 10, 2021, 6:17am UTC](https://discourse.julialang.org/t/ann-juliasmoothoptimisers-just-got-smoother-reverse-mode-ad-support/58966/1 "2021-04-10T06:17:43Z")

</div>

I am glad to announce that you can now use reverse-mode automatic differentiation (AD) packages such as Zygote and ReverseDiff in the JuliaSmoothOptimisers ecosystem. This is part of an effort towards an AD-based first-order augmented Lagrangian framework which is close but not there yet. Here is the syntax for defining a model that makes use of Zygote:

```julia
using ADNLPModels, LinearAlgebra, Zygote

x0 = zeros(2)
f(x) = dot(x,x)
nlp = ADNLPModel(f, x0, adbackend = ADNLPModels.ZygoteAD())
x = ones(2)
ADNLPModels.grad(nlp, x) == [2, 2]

```

Happy optimisation!

---

<div class="post-metadata">

**Author:** ![richinex](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/richinex/32/43805_2.png) [@richinex](https://discourse.julialang.org/u/richinex)\
**Post date:** [October 26, 2022, 4:25pm UTC](https://discourse.julialang.org/t/ann-juliasmoothoptimisers-just-got-smoother-reverse-mode-ad-support/58966/2 "2022-10-26T16:25:09Z")

</div>

> [@mohamed82008](#):
>
> ```julia
> using ADNLPModels, LinearAlgebra, Zygote
> 
> x0 = zeros(2)
> f(x) = dot(x,x)
> nlp = ADNLPModel(f, x0, adbackend = ADNLPModels.ZygoteAD())
> x = ones(2)
> ADNLPModels.grad(nlp, x) == [2, 2]
> 
> ```

I get the error below when I try to run this example

UndefVarError: ZygoteAD not defined

---

<div class="post-metadata">

**Author:** ![mohamed82008](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mohamed82008/32/18171_2.png) [@mohamed82008](https://discourse.julialang.org/u/mohamed82008)\
**Post date:** [October 26, 2022, 4:34pm UTC](https://discourse.julialang.org/t/ann-juliasmoothoptimisers-just-got-smoother-reverse-mode-ad-support/58966/3 "2022-10-26T16:34:56Z")

</div>

The syntax changed multiple times since the announcement. You basically have to do something like this now which is a little more verbose:

```julia
ZygoteAD() = ADNLPModels.ADModelBackend(
  ADNLPModels.ZygoteADGradient(),
  ADNLPModels.ForwardDiffADHvprod(),
  ADNLPModels.ZygoteADJprod(),
  ADNLPModels.ZygoteADJtprod(),
  ADNLPModels.ZygoteADJacobian(0),
  ADNLPModels.ZygoteADHessian(0),
  ADNLPModels.ForwardDiffADGHjvprod(),
)
nlp = ADNLPModel(f, x0)
set_adbackend!(nlp, ZygoteAD())

```

---

<div class="post-metadata">

**Author:** ![tmigot](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tmigot/32/23914_2.png) [@tmigot](https://discourse.julialang.org/u/tmigot)\
**Post date:** [October 26, 2022, 4:49pm UTC](https://discourse.julialang.org/t/ann-juliasmoothoptimisers-just-got-smoother-reverse-mode-ad-support/58966/4 "2022-10-26T16:49:26Z")

</div>

Hi @richinex

The syntax has changed a bit so that ADNLPModel becomes more flexible.  
You can have a look at the documentation here: [Backend · ADNLPModels.jl](https://juliasmoothoptimizers.github.io/ADNLPModels.jl/stable/backend/)

The following should work:

```julia
using ADNLPModels, LinearAlgebra, Zygote

x0 = zeros(2)
f(x) = dot(x,x)
nlp = ADNLPModel(f, x0, gradient_backend = ADNLPModels.ZygoteADGradient)
x = ones(2)
ADNLPModels.grad(nlp, x) == [2, 2]

```

---

<div class="post-metadata">

**Author:** ![mohamed82008](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mohamed82008/32/18171_2.png) [@mohamed82008](https://discourse.julialang.org/u/mohamed82008)\
**Post date:** [October 26, 2022, 4:57pm UTC](https://discourse.julialang.org/t/ann-juliasmoothoptimisers-just-got-smoother-reverse-mode-ad-support/58966/5 "2022-10-26T16:57:59Z")

</div>

I realised my initial suggestion doesn’t actually end up using Zygote, so I made an edit. But I would use @tmigot’s suggestion anyways since he is a core dev of the package and I am not.

---

<div class="post-metadata">

**Author:** ![richinex](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/richinex/32/43805_2.png) [@richinex](https://discourse.julialang.org/u/richinex)\
**Post date:** [October 26, 2022, 5:29pm UTC](https://discourse.julialang.org/t/ann-juliasmoothoptimisers-just-got-smoother-reverse-mode-ad-support/58966/6 "2022-10-26T17:29:41Z")

</div>

Cool. Thanks