We are wrapping up the API for a GSOC project and I need some help about how to best integrate this into the existing Julia ecosystem.
From a theoretical point of view, the API needs a simple function ℓ:ℝⁿ→ℝ (log posterior), its gradient ∇ℓ:ℝⁿ→ℝⁿ, and a starting point θ. For the first two, the user currently supplies a model object model, which, among other things (bookkeeping), mostly supports logdensity(model, θ) and loggradient(model, θ). But since both the gradient and the density are frequently needed together, this is wasteful, and something like the DiffResult API could be a good choice. Important: type stability is crucial.
For the actual algorithm, possibly ForwardDiff.jl or ReverseDiff.jl may be used, or some other option (eg manual calculation of derivatives for some simple models). I don’t want the API to commit to either. This should be up to the user.
What I thought about: have an interface which supports
gradient!(result::DiffBase.DiffResult, model, θ)
for model and θ. When defining this method, the user picks the implementation (forward or reverse AD, or manual calculation). Choices and parameters specific to this could be shoved into model (eg ForwardDiff.GradientConfig). The interface calls gradient!, then queries result for value and gradient.
Problem: all methods would need to pass around result in addition to model all the way down. Possible solution: wrap result and model in a single object together, but this looks inelegant.
It is possible that I am not grasping the related APIs well. Any advice would be appreciated.