# TransformerBlocks.jl - Simple, blazing fast, transformer components

**URL:** <https://discourse.julialang.org/t/transformerblocks-jl-simple-blazing-fast-transformer-components/96266>\
**Category:** Package Announcements\
**Tags:** machine-learning\
**Created:** [March 17, 2023, 11:07pm UTC](https://discourse.julialang.org/t/transformerblocks-jl-simple-blazing-fast-transformer-components/96266 "2023-03-17T23:07:30Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![JuliaMLTools](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juliamltools/32/47838_2.png) [@JuliaMLTools](https://discourse.julialang.org/u/JuliaMLTools)\
**Post date:** [March 17, 2023, 11:07pm UTC](https://discourse.julialang.org/t/transformerblocks-jl-simple-blazing-fast-transformer-components/96266/1 "2023-03-17T23:07:30Z")

</div>

Hello everyone!

I’m very happy to publish my first package:

> **[GitHub - JuliaMLTools/TransformerBlocks.jl: Simple, blazing fast, transformer...](https://github.com/JuliaMLTools/TransformerBlocks.jl)**
>
> Simple, blazing fast, transformer components. Contribute to JuliaMLTools/TransformerBlocks.jl development by creating an account on GitHub.

In attempting to create the most performant implementation of Andrej Karpathy’s excellent ShakespeareGPT example (as seen on YouTube), I decided to publish my work as a package. I hope some of you find it useful.

The example code using TransformerBlocks.jl to implement ShakespeareGPT can be found here:  
[https://juliamltools.github.io/shakespeare-gpt](https://juliamltools.github.io/shakespeare-gpt)

I’d love to get some real, critical feedback on my code style and to know how I can improve to better fit in with the Julia ecosystem. I really enjoy Julia and intend on releasing many more projects and tools. Thanks for your time.

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [March 18, 2023, 8:28am UTC](https://discourse.julialang.org/t/transformerblocks-jl-simple-blazing-fast-transformer-components/96266/2 "2023-03-18T08:28:26Z")

</div>

Great idea, thanks for the contribution!  
Just copying what was said on Slack before it gets erased: to achieve top performance, it’s a good idea to [profile your code](https://www.julia-vscode.org/docs/stable/userguide/profiler/) and see which parts take longest. At first glance, it seems like your [untyped struct fields](https://docs.julialang.org/en/v1/manual/performance-tips/#man-performance-abstract-container) might result in type-instability.  
If you need help solving that or other issues, I’m happy to contribute 🙂

---

<div class="post-metadata">

**Author:** ![JuliaMLTools](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juliamltools/32/47838_2.png) [@JuliaMLTools](https://discourse.julialang.org/u/JuliaMLTools)\
**Post date:** [March 18, 2023, 12:32pm UTC](https://discourse.julialang.org/t/transformerblocks-jl-simple-blazing-fast-transformer-components/96266/3 "2023-03-18T12:32:52Z")

</div>

Thanks! Just updated. Look OK?

> <https://github.com/JuliaMLTools/TransformerBlocks.jl/commit/76b6eade5869dce882e07f8fc63de5d52538e53d>

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [March 18, 2023, 5:07pm UTC](https://discourse.julialang.org/t/transformerblocks-jl-simple-blazing-fast-transformer-components/96266/4 "2023-03-18T17:07:53Z")

</div>

Looks good to me! I haven’t checked the rest but this is already a major step 👍

---

<div class="post-metadata">

**Author:** ![bertschi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bertschi/32/33462_2.png) [@bertschi](https://discourse.julialang.org/u/bertschi)\
**Post date:** [March 19, 2023, 9:24am UTC](https://discourse.julialang.org/t/transformerblocks-jl-simple-blazing-fast-transformer-components/96266/5 "2023-03-19T09:24:03Z")

</div>

Nice work and very clean/readable code. Typing the struct fields has also been done already.  
From a bigger perspective, I wonder how your library compares to [Transformers.jl](https://github.com/chengchingwen/Transformers.jl) and [NNlib.jl](https://github.com/FluxML/NNlib.jl)? Are you planning to complement or extend them? Is your implementation faster?

---

<div class="post-metadata">

**Author:** ![JuliaMLTools](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juliamltools/32/47838_2.png) [@JuliaMLTools](https://discourse.julialang.org/u/JuliaMLTools)\
**Post date:** [March 19, 2023, 3:32pm UTC](https://discourse.julialang.org/t/transformerblocks-jl-simple-blazing-fast-transformer-components/96266/6 "2023-03-19T15:32:34Z")

</div>

TransformerBlocks.jl is more than 10 times faster than Transformers.jl in this example:  
[https://juliamltools.github.io/shakespeare-gpt](https://juliamltools.github.io/shakespeare-gpt)

The latest I heard from the author of Transformers.jl is that some type instability issues may be causing issues there.

I haven’t yet compared TransformerBlocks.jl with NNlib.jl.

Thanks for the feedback!