# What lessons could Julia's autodiff ecosystem learn from Stan's TinyGrad?

**URL:** <https://discourse.julialang.org/t/what-lessons-could-julias-autodiff-ecosystem-learn-from-stans-tinygrad/103070>\
**Category:** Machine Learning\
**Created:** [August 22, 2023, 2:50pm UTC](https://discourse.julialang.org/t/what-lessons-could-julias-autodiff-ecosystem-learn-from-stans-tinygrad/103070 "2023-08-22T14:50:56Z")\
**Posts on this page:** 1\
**Showing post:** 22

<div class="post-metadata">

**Author:** ![bertschi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bertschi/32/33462_2.png) [@bertschi](https://discourse.julialang.org/u/bertschi)\
**Post date:** [August 22, 2023, 6:54pm UTC](https://discourse.julialang.org/t/what-lessons-could-julias-autodiff-ecosystem-learn-from-stans-tinygrad/103070/22 "2023-08-22T18:54:17Z")

</div>

> [@stevengj](#):
>
> Another alternative is to create a walled garden like JAX, in which the only things that are likely to work are compositions of `jax.*`. Don’t use Numpy, use `jax.numpy`. Don’t use Scipy, use `jax.scipy`. Use `jax.experimental.ode.odeint` to integrate your ODEs, and so forth.

To me every Python AD feels like a walled garden as each comes with its own incompatible tensor types and slightly different (sub)set of supported operations. Some of these gardens are just very large by now, e.g., PyTorch is aiming to reduce its set of operators from 2000+ to around 250 core ones.  
In Julia, I can just write my model – possibly combining several libraries – and then try different ADs on it. Especially `AbstractDifferentiation` makes it very easy to change the AD backend. In my experience, `ForwardDiff` and `ReverseDiff` have worked quite reliable (even in some [crazy use cases](https://discourse.julialang.org/t/seven-lines-of-julia-examples-sought/50416/144) that I would not have tried in Python to begin with). I did face several issues with Zygote though, either hitting some limitations or silently wrong gradients, so I tend to avoid it for now.

---

_[View the full topic](https://discourse.julialang.org/t/what-lessons-could-julias-autodiff-ecosystem-learn-from-stans-tinygrad/103070)._
