Understand this part of the Zygote docs

Specific Domains Machine Learning
1

I am wondering if there are several typos in introduction example to custom adjoints,
https://fluxml.ai/Zygote.jl/dev/adjoints/

In other words pullback(sin, x) behaves the same as

dsin(x) = sin(x), ȳ -> (ȳ * cos(x),)

Is the dsin(x) = sin(x) correct? Does this mean that pullback(sin,x) returns a tuple containing 1) an new aliased function for sin, 2) a function that returns whatever its argument is times cos(x)?

Then the next example says

gradient takes a function l=f(x) and … feeds this in to the pullback. In the case of sin ,

julia> function gradsin(x)
         _, back = dsin(x)
         back(1)
       end

Multiple strange things here (to me). 1) pullback is not called anywhere,
2) dsin(x) = sin(x), so back is equal to sin(0.5), and is not a function (so back(1) should give an error)

Are there any more clear documentation?

2

That is not what it says, adding the redundant parenthesis might show this more clearly:

dsin(x) = (sin(x), ȳ -> (ȳ * cos(x),))

In other words, dsin(x) returns a tuple with the value of sin(x) as the first element and an anonymous function that does the derivative propagation (chain rule).

1 Like
3

thank you