We are wrapping up the API for a GSOC project and I need some help about how to best integrate this into the existing Julia ecosystem.
From a theoretical point of view, the API needs a simple function
ℓ:ℝⁿ→ℝ (log posterior), its gradient
∇ℓ:ℝⁿ→ℝⁿ, and a starting point
θ. For the first two, the user currently supplies a model object
model, which, among other things (bookkeeping), mostly supports
logdensity(model, θ) and
loggradient(model, θ). But since both the gradient and the density are frequently needed together, this is wasteful, and something like the DiffResult API could be a good choice. Important: type stability is crucial.
For the actual algorithm, possibly
ReverseDiff.jl may be used, or some other option (eg manual calculation of derivatives for some simple models). I don’t want the API to commit to either. This should be up to the user.
What I thought about: have an interface which supports
gradient!(result::DiffBase.DiffResult, model, θ)
θ. When defining this method, the user picks the implementation (forward or reverse AD, or manual calculation). Choices and parameters specific to this could be shoved into
ForwardDiff.GradientConfig). The interface calls
gradient!, then queries
Problem: all methods would need to pass around
result in addition to
model all the way down. Possible solution: wrap
model in a single object together, but this looks inelegant.
It is possible that I am not grasping the related APIs well. Any advice would be appreciated.