# Factor of two in Zygote complex gradient

**URL:** <https://discourse.julialang.org/t/factor-of-two-in-zygote-complex-gradient/77545>\
**Category:** Optimization (Mathematical)\
**Tags:** zygote\
**Created:** [March 7, 2022, 11:51pm UTC](https://discourse.julialang.org/t/factor-of-two-in-zygote-complex-gradient/77545 "2022-03-07T23:51:17Z")\
**Posts on this page:** 1\
**Showing post:** 3

<div class="post-metadata">

**Author:** ![goerz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/goerz/32/3269_2.png) [@goerz](https://discourse.julialang.org/u/goerz)\
**Post date:** [March 8, 2022, 2:41am UTC](https://discourse.julialang.org/t/factor-of-two-in-zygote-complex-gradient/77545/3 "2022-03-08T02:41:58Z")

</div>

In the real case, I’d agree, but for a complex vector, I don’t think that’s still true. In fact, it’s the same for a complex scalar: For `J(z) = 1 - |z|² = 1 - z z̄`, the Wirtinger derivative `∂J/∂z̄` would be defined as `(∂J/∂Re[z] + 𝕚 ∂J/∂Im[z])/2`, see e.g. [Complex Derivatives, Wirtinger View and the Chain Rule | Ekin Akyürek](https://www.ekinakyurek.me/complex-derivatives-wirtinger/). So, with `z = x + 𝕚y` that would be `-(2x + 𝕚 2y) / 2 = -z`, which also matches the intuition that `z` and `z̄` can simply be treated as independent variables. However, Zygote gives

```julia
J(z) = 1 - abs(z)^2
Zygote.gradient(z -> J(z), 1+1im)
(-2.0 - 2.0im,)

```

I then found that the Zygote manual uses _exactly_ this example in the [chapter on Complex Differentiation](https://fluxml.ai/Zygote.jl/latest/complex/), and that I just didn’t read that closely enough. It also gives the solution of manually defining a Wirtinger derivative. This indeed also works for the vector case. In my example script, I can add

```julia
# Wirtinger derivative
function wirtinger(func, Ψ)
    y, back = Zygote.pullback(func, Ψ)
    du = back(1.0)[1]
    dv = back(1.0im)[1]
    return (du' + im*dv')/2, (du + im*dv)/2
end

∇3 = wirtinger(Ψ -> J(Ψ, Ψtgt), Ψ)[2]
norm(abs.(∇3 - ∇2)) # zero, as it should be

```

which does the right thing.

I’m still a little confused about what just `Zygote.gradient` does (I might have to re-read that chapter a few more times). In the scalar case, I’m pretty sure it does

```julia
∂f(z)/∂z -> ∂f(x, y)/∂x + 𝕚 ∂f(x, y)/∂y

```

which seems somewhat trivially connected to the correct Wirtinger derivative (up to my missing factor of 2).

On the other hand, it would seem like `Zygote.gradient` does a single backward-propagation, whereas `wirtinger` does two. So probably, I can’t use the simple `Zygote.gradient` in general.

---

_[View the full topic](https://discourse.julialang.org/t/factor-of-two-in-zygote-complex-gradient/77545)._