# LlaMa2 architecture in Julia: llama2.jl (300 lines?)

**URL:** <https://discourse.julialang.org/t/llama2-architecture-in-julia-llama2-jl-300-lines/102006>\
**Category:** General Usage\
**Created:** [July 24, 2023, 11:33am UTC](https://discourse.julialang.org/t/llama2-architecture-in-julia-llama2-jl-300-lines/102006 "2023-07-24T11:33:32Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jose\_Diaz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jose_diaz/32/43750_2.png) [@Jose\_Diaz](https://discourse.julialang.org/u/Jose_Diaz)\
**Post date:** [July 24, 2023, 11:33am UTC](https://discourse.julialang.org/t/llama2-architecture-in-julia-llama2-jl-300-lines/102006/1 "2023-07-24T11:33:32Z")

</div>

Hello!

I’m currently porting [karpathy/llama2.c](https://github.com/karpathy/llama2.c) to Julia  
Managed to import all the weights and the config. Created structs with the appropiate types and implemented some of the math too. Now I’m implementing the architecture.  
I’m having some issues to understand the way arrays are sliced in C but I’m managing I think

You can check out the progress here: [GitHub - jdiaz97/llama2.jl: llama2 in Julia](https://github.com/jdiaz97/llama2.jl)  
I’m trying to make a rough port first and once it works I’ll refactor it to be more julian and fast (loop vectorization, parallelization)  
The final version should have way less lines, be more readable and faster than karpathy’s C version.

---

<div class="post-metadata">

**Author:** ![pitsianis](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pitsianis/32/26588_2.png) [@pitsianis](https://discourse.julialang.org/u/pitsianis)\
**Post date:** [August 1, 2023, 5:21am UTC](https://discourse.julialang.org/t/llama2-architecture-in-julia-llama2-jl-300-lines/102006/2 "2023-08-01T05:21:27Z")

</div>

This one is already up and running

> **[GitHub - cafaxo/Llama2.jl: llama2.c but in Julia](https://github.com/cafaxo/Llama2.jl)**
>
> llama2.c but in Julia. Contribute to cafaxo/Llama2.jl development by creating an account on GitHub.

in case you have not seen it yet.

---

<div class="post-metadata">

**Author:** ![ericphanson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ericphanson/32/215186_2.png) [@ericphanson](https://discourse.julialang.org/u/ericphanson)\
**Post date:** [August 1, 2023, 5:59am UTC](https://discourse.julialang.org/t/llama2-architecture-in-julia-llama2-jl-300-lines/102006/3 "2023-08-01T05:59:58Z")

</div>

There is also

> **[GitHub - rai-llc/LanguageModels.jl: Load nanoGPT-style transformers in Julia....](https://github.com/rai-llc/LanguageModels.jl)**
>
> Load nanoGPT-style transformers in Julia. Code ported from @karpathy's llama2.c - GitHub - rai-llc/LanguageModels.jl: Load nanoGPT-style transformers in Julia. Code ported from @karpathy's ...