Each AD gets given the full expression graph, along with callbacks for how to numerically evaluate the function, gradient, a hessian of each operator. Then they have to produce a MOI.AbstractNLPEvaluator which implements the MOI callbacks like eval_constraint_jacobian and MOI.hessian_lagrangian_structure.
We have a few implementations already which gives us confidence this is a reasonable thing to do:
- The
SparseReverseADin JuMP SymbolicADin GitHub - odow/MathOptSymbolicAD.jl, which uses Symbolics.jl to compute sparse derivatives, but which uses some tricks to avoid computing the symbolic derivative of the full problemMadDiff: GitHub - sshin23/MadDiff.jl: An automatic differentiation and algebraic modeling package
For @ccoffrin’s AC-OPF problems, SymbolicAD is 3-5x faster than SparseReverseAD, but on other problems it can be much worse.