Hi all,
I have been leading the ChainRules project since juliacon 2019.
I find myself in circumstances very similar to what Jarret was when he transferred leadership of it to me then – this is a very good thing.
As such I will have less time to devote towards it.
The ChainRules project has been all in all successful.
It is supported (to some extent) by 10 different julia AD engines, and it has hundreds of rules.
Its not entirely up to the latest needs of all engines but it very widely used.
Hundreds of direct dependencies, thousands of indirect dependencies.
The project is now pretty stable.
So the main things now for a lead maintainer to do are:
Triage issues: go through new issues as they are openned and give them labels, and check they are not results of confusions, and encourage people to make PRs.
Review PRs: review the PRs being made and make the final call, or tag in other maintainers to do reviews
Optionally Plan and implement big changes: sometimes we need to support new things for AD engines (most recently was mutating forwards mode), but we don’t always need to do that
We do have numerous other contributors, and I have spoken to several of them (but others I could not reach) and most of them are also restricted in their time.
As such I post here
Ideally after Juliacon this year, I will unfollow the ChainRulesCore, ChainRules, ChainRulesTestUtils and possibly FiniteDifferences repos and only respond when pinged.
I am happy to respond when pinged, to explain things.
And more broadly talk a new maintainer through various things.
First of all a big “thank you” for your contribution, ChainRules.jl is one the packages that make Julia so attractive compared to more traditional machine learning pipelines such as python/pytorch.
ChainRules.jl is now a very important package for the Julia community and I am a little bit concerned of the potential impact of its slowing down would have on the julia AD ecosystem. Personally, my research is heavily relying on the Julia AD capability (ChainRules, Flux, Zygote, and hopefully Diffractor when reverse mode AD is fully working) and if these packages cannot be maintained on the long run that would be a huge blow for me (and many others I believe).
As you have been a key actor of the Julia AD ecosystem I would like to ask whether you feel Julia AD environment is still active and evolving or is it at risk of lacking supports to still be up to date and innovative ?
Thank you for your consideration and I reiterate my appreciation to your contribution to the community.
There are definitely other people still making active strides on various parts of the ecosystem.
@wsmoses is pushing out new Enzyme.jl features all the time
@gdalle has been making regular improvements in DifferentationInterface.jl
@willtebbutt recently released a new reverse mode AD Tapir.jl.
And I am sure there are things i am not aware of.
Its notable that the two AD systems i mentioned as being the ones I am aware of as most active only have limited dependencies on ChainRules for their ecosystem.
They support importing rules from ChainRules (with various crevats), but that is less powerful than their own rule systems.
And I do know they sometimes have ported rules out of ChainRules – using it as a informational, rather than programatic, resource.
@lucas.ondel_yang – I would second what @oxinabox says – reverse-mode AD in Julia is definitely still being actively worked on!
On Tapir.jl (which is soon to be renamed to something else due to a conflict), it’s actually pretty high up our priority list to impove our interop with ChainRules.jl – at the minute it’s very limited, but we think it can be generalised substantially.
As @oxinabox says though, both Tapir.jl and Enzyme.jl are much less dependent on ChainRules.jl’s rules, because they need fewer rules, because they’re both much better than Zygote.jl at differentiating Julia code in a performant manner (many rrules are written to work around Zygote.jl’s limitations).
Also, I would just add that you’re certainly not alone in being strongly reliant on reverse-mode AD in your work. I work closely with / am part of the Turing.jl team, and reverse-mode AD is completely essential for our work also, so we’re strongly incentivised to ensure that the reverse mode AD in Julia continues to improve.
Indeed there was a thread the other day Enzyme ready for everyday use? (2024) - #12 by ChrisRackauckas where it seems like many reverse mode AD users are starting to shift towards Enzyme – which is quite actively developed and now has reasonable coverage of generic julia code.
Give it a try at let me know if something doesn’t work?
And to complete the trinity, I am indeed actively developing DifferentiationInterface.jl. In particular, our sparse Jacobians and Hessians are pretty much state of the art even compared to Python (thanks to @hill and @amontoison).
Recently there has been a lot of change in the background, and now I am almost ready for a breaking release which will affect the argument order. Once that is done, I’ll be able to support a long-awaited feature: multiple arguments with different activities. This will allow users to make the most of Enzyme’s abilities, which thanks to Billy are evolving fast.
If you’re an AD client wondering which package to use in your work, or if you don’t want to commit right away, I will gladly help you set up DI, just ping me here or on GitHub.
