For interest, and not something you should action:
adjoint in this context means “Method to propagate the derivative backwards” (in this case through the constructor).
Its the pullback or vector transpose jacobian product or v’jp (sometimes written vjp and called vector jacobian product. Or j’vp Jacobian transpose vector product)
Its actually closely related to the adjoint of a matrix.
Lets see if i can describe this right:
- If one linearizes the computer program at a point, then the program becomes a linear map,
- linearlizing the program means findind program that approximates the real programs output using tangent plane
- estimates how much a change in the input would change the outpuit
- forward mode AD does this linearizing op
- linear maps can also be represented as matrixes
- that matrix would be the jacobian.
- so linearlized functions are the function form of the jacobian matrix.
- Now if one takes a transpose or adjoint of the jacobian what does that do?
- (there is a ever present debate as to if one should be taking the conjugate or not. Complex number AD is a contentious subfield.)
- it would take in a change in output and (under a linear approximation) estimate what change in input would cause it.
- This is what reverse mode AD does too
- Thus link forward mode AD gives a function form of the Jacobain. reverse mode gives its transpose (or adjoint)
- thus reverse mode is findin the (linear) adjoint program.
And flux is complaining that for the operation: constructor SparseMatrixCSC{Float64,Int64}.
zygote doesn’t know how to find the adjoint program.
basically Zygote has trouble autodiffing through constructors and needs to be told explictly how to do it. (I’ld like to look into why that is, I don’t myself know I haven’t looked at that part of the source)
It needs a custom rule written, which it has many custom rules already but that one I guess was missed.
The old AD system tracker also had trouble with this but for very different reasons, and it already had the fix in place. (and the fix looked very different because it came from different reasons. So I see why it was missed)
That particular one is odd though,
Because the