What's the state of Automatic Differentiation in Julia January 2023?

I mean it’s probably useful to mixed mode broadcast regardless.

However, what that PR does is generically say that autodiff of @cuda is @cuda of autodiff (which is presently needed by broadcasting among other things).

It’s definitely useful to also consider what higher level utilities we want to add – but that is useful as a baseline so generic code doesn’t need to pipe in an inner autodiff call inside all @cuda’s.

That’s why I’ll argue that the linked PR makes broadcast work in reverse mode (among other things), but will potentially get further improved with additional broadcast-specific tuning.

1 Like