# \[ANN\] Jjama3.jl (unregistered) - Llama3.1 and Llama3.2 (text) in Julia

**URL:** <https://discourse.julialang.org/t/ann-jjama3-jl-unregistered-llama3-1-and-llama3-2-text-in-julia/122641>\
**Category:** Package Announcements\
**Created:** [November 14, 2024, 4:15pm UTC](https://discourse.julialang.org/t/ann-jjama3-jl-unregistered-llama3-1-and-llama3-2-text-in-julia/122641 "2024-11-14T16:15:22Z")\
**Posts on this page:** 1\
**Showing post:** 4

<div class="post-metadata">

**Author:** ![Noob](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/noob/32/6595_2.png) [@Noob](https://discourse.julialang.org/u/Noob)\
**Post date:** [November 26, 2024, 11:54pm UTC](https://discourse.julialang.org/t/ann-jjama3-jl-unregistered-llama3-1-and-llama3-2-text-in-julia/122641/4 "2024-11-26T23:54:54Z")

</div>

A few additions to this today:

- Life is too short to spend on tokenizers, so we switched to using [a julia wrapper of a python wrapper of Hugging Face’s rust tokenizers library](https://github.com/MurrellGroup/HuggingFaceTokenizers.jl) (hat tip @AntonOresten for this!).
- Added low-rank (LoRA) finetuning.
- Added new samplers, including classics top-p & top-k, but also the more recent [https://arxiv.org/pdf/2407.01082](https://arxiv.org/pdf/2407.01082) and [https://arxiv.org/pdf/2411.07641](https://arxiv.org/pdf/2411.07641)
- In response to a [Bluesky conversation about structured sampling](https://bsky.app/profile/cscherrer.bsky.social/post/3lbsacrbrgm2w) we added an example of a sampler that lets you restrict the logits to match a custom template.

Switching tokenizers unlocked some fun small open models, like the SmolLM2 series (more open than Llama3.2, which is behind a permissions wall, so this might reduce a barrier to getting started). With the LoRA addition, this is at a fairly decent point for someone wanting to tinker with LLMs. Cooking up new samplers is a fun sport (evaluating them is trickier), and you can finetune a 1.7 billion parameter model just on your CPU (see our example where we make one much stupider).

---

_[View the full topic](https://discourse.julialang.org/t/ann-jjama3-jl-unregistered-llama3-1-and-llama3-2-text-in-julia/122641)._