# Simplechains.jl vs. George Hotz & Tinygrad?

**URL:** <https://discourse.julialang.org/t/simplechains-jl-vs-george-hotz-tinygrad/90387>\
**Category:** Machine Learning\
**Tags:** question\
**Created:** [November 17, 2022, 7:09am UTC](https://discourse.julialang.org/t/simplechains-jl-vs-george-hotz-tinygrad/90387 "2022-11-17T07:09:36Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![artkuo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/artkuo/32/28577_2.png) [@artkuo](https://discourse.julialang.org/u/artkuo)\
**Post date:** [November 17, 2022, 7:09am UTC](https://discourse.julialang.org/t/simplechains-jl-vs-george-hotz-tinygrad/90387/1 "2022-11-17T07:09:36Z")

</div>

George Hotz recently stepped down from comma.ai and [announced](https://geohot.github.io//blog/jekyll/update/2022/10/29/the-heroes-journey.html) he may devote more effort to his Tinygrad package:

> I’m considering another company, **the Tiny Corporation**. Under 1000 lines, under 3 people, 3x faster than PyTorch? For smaller models, there’s so much left on the table. And if you step away from the well-tread ground of x86 and CUDA, there’s 10x+ performance to gain. Several very simple abstractions cover all modern deep learning, today’s libraries are way too complex.

Superficially, this sounds a bit like the goals of [SimpleChains.jl](https://julialang.org/blog/2022/04/simple-chains/). Question is how SimpleChains.jl differs in approach and goals from his [Tinygrad](https://github.com/geohot/tinygrad). The speed improvements vs. PyTorch sound pretty comparable. I think both are currently CPU only, and in short term Tinygrad may support Apple Silicon and Google TPU, and long term they want to do their own hardware. What do people think, will SimpleChains exceed it?

---

<div class="post-metadata">

**Author:** ![tchebycheff](https://avatars.discourse-cdn.com/v4/letter/t/779978/32.png) [@tchebycheff](https://discourse.julialang.org/u/tchebycheff)\
**Post date:** [November 27, 2022, 12:36pm UTC](https://discourse.julialang.org/t/simplechains-jl-vs-george-hotz-tinygrad/90387/2 "2022-11-27T12:36:29Z")

</div>

I agree with their statement that 90% of what is required is just an efficient way to calculate gradients. Perhaps, Julia could have AD as part of standard library (apart from json, csv and http handling).

---

<div class="post-metadata">

**Author:** ![Elrod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/elrod/32/22461_2.png) [@Elrod](https://discourse.julialang.org/u/Elrod)\
**Post date:** [November 27, 2022, 7:18pm UTC](https://discourse.julialang.org/t/simplechains-jl-vs-george-hotz-tinygrad/90387/3 "2022-11-27T19:18:11Z")

</div>

Would be interesting to compare both on MNIST.

My long term plan for SimpleChains is for it to be based on LoopModels + Enzyme.

For now, it is LoopVectorization.jl + pull back definitions.  
Memory management is manual, and we should create a better API for that with cleaner separation.