KeemenaLM.jl v0.1.0, a Julia proof-of-concept for training and running a small GPT-style chatbot from scratch.
It includes a v9 broad 336M scratch-trained chatbot baseline as a GitHub Release artifact. The model is a research baseline, not a reliable assistant yet, but the Julia training, export, artifact, tokenizer, and REPL path are working.
using Pkg
Pkg.add(url = "https://github.com/mantzaris/KeemenaLM.jl", rev = "v0.1.0")
using KeemenaLM
bundle_dir = download_model("tiny-chatbot-v9-broad-336m")
tokenizer_dir = resolve_tokenizer_bundle("tiny-chatbot-v9-broad-336m")
It also used other Julia packages such as
https://github.com/mantzaris/KeemenaSubwords.jl
and
https://github.com/mantzaris/KeemenaPreprocessing.jl
I would be happy to talk with others interested in Julia for LLMs and work together to improve the training to make a better chatbot, or some other application even.