What lessons could Julia's autodiff ecosystem learn from Stan's TinyGrad?

[ParadaCarleton]

@dlakelan

No, maybe more like 3-4 years, and with Bob Carpenter a major driver of design and implementation, and several million dollars of grant funds.

This was inspired by a conversation with Bob about this where he cited that it:

involved about 2 person years of effort spread over 18 months. We did it on two $60K postdoc salaries and one $40K TA salary.

@dlakelan

If someone plops $5M into a group of 5 people to finish up milestones for Enzyme you’d see a lot of progress in the next 3 years

Right, the problem is it costs $5M and takes a group of 5 people 3 years.

@aplavin

As for autodiff, I want to really appreciate how ForwardDiff works in Julia!

1,000,000,000%. I am completely in love with ForwardDiff and it’s by far my favorite autodiff tool in Julia. The only problem is, as you mentioned, that it’s forward-mode

@stevengj

This is much, much easier than something like Enzyme that aims to AD arbitrary code.

Exactly!