LogDensityProblems, for example, while I like it, does have it’s own opinions on how to run AD frameworks and has a load time of 7.5s
@Tamas_Papp: The load time is coming from Requires.jl, which is an open issue. Hopefully that will be fixed in the long run
Sure, I know - but the long run might be quite a while yet …
I am still not yet convinced about the benefits of this. I understand that it can be nice to run optimization using different frameworks with a single API, and I think that the optimization packages around Optim.jl have a common interface for that already
Exactly - I think the approach about having lightweight common “almost-API-only” packages has been extremely successful in the Julia community/ecosystem. We have plotting recipes, we have ZygoteRules and ChainRulesCore, we have NLSolversBase and MathOptInterface, and so on.
So why wouldn’t we profit from having a common API for target functions like densities and loss functions, so that these can be coded up in a way that’s compatible with a variety of algorithm implementations?
Different package authors have different styles, and it may be difficult to cover everything.
Certainly, but I think something like @ChrisRackauckas proposed
[…] interface be on functions, i.e.
value(res)instead of on properties, i.e.res.value.
and like you’re planning yourself with get_logdensity, get_gradient and get_payload could probably go quite a long way in this case.
I’m not talking about a common API for samplers, etc. Just a common API to access parts of the return value of target/loss functions: The log(value), optionally a gradient and optionally auxiliary information (what you called a payload).
If you are already thinking about get_logdensity, get_gradient and get_payload (whatever they will be named), surely it would be beneficial to have these in a lightweight central package with almost zero load time and dependencies? Certainly there would be no harm in it. And if we could get packages like (e.g.) DynamicHMC, AdvancedHMC, Optim, etc. to support that interface - maybe in addition to their own style - I think users and generic code that implements denstiy/loss-functions could potentially benefit a lot from that.
It would be nice if in the easy case, the user would just have to implement a simple function/closure, with no special methods, and could just return a (logd = ..., [gradient = ...]) - which get_density_logval and friends would know how to handle by default. And in more complex cases anything could be returned, as long as the access-functions get methods for that custom type.