# Proof of concept LLM chatbot built with Julia: KeemenaLM.jl

**URL:** <https://discourse.julialang.org/t/proof-of-concept-llm-chatbot-built-with-julia-keemenalm-jl/137961>\
**Category:** Package Announcements\
**Created:** [July 5, 2026, 8:37pm UTC](https://discourse.julialang.org/t/proof-of-concept-llm-chatbot-built-with-julia-keemenalm-jl/137961 "2026-07-05T20:37:34Z")\
**Posts on this page:** 12\
**Page:** 3

<div class="post-metadata">

**Author:** ![j\_u](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/j_u/32/219081_2.png) [@j\_u](https://discourse.julialang.org/u/j_u)\
**Post date:** [August 21, 2026, 7:54am UTC](https://discourse.julialang.org/t/proof-of-concept-llm-chatbot-built-with-julia-keemenalm-jl/137961/41 "2026-08-21T07:54:41Z")

</div>

I particularly like the ones about long context windows but not only.

---

<div class="post-metadata">

**Author:** ![Myar\_joyia](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/myar_joyia/32/223456_2.png) [@Myar\_joyia](https://discourse.julialang.org/u/Myar_joyia)\
**Post date:** [August 21, 2026, 11:24am UTC](https://discourse.julialang.org/t/proof-of-concept-llm-chatbot-built-with-julia-keemenalm-jl/137961/42 "2026-08-21T11:24:34Z")

</div>

That makes sense. The long-context work is especially interesting because it seems closely connected to where RLMs are heading. I’d also be curious to see whether some of these ideas could be practically explored in Julia, particularly around recursive inference and context management. That could make for a really interesting direction for the primer and the project overall.

---

<div class="post-metadata">

**Author:** ![j\_u](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/j_u/32/219081_2.png) [@j\_u](https://discourse.julialang.org/u/j_u)\
**Post date:** [August 21, 2026, 3:21pm UTC](https://discourse.julialang.org/t/proof-of-concept-llm-chatbot-built-with-julia-keemenalm-jl/137961/43 "2026-08-21T15:21:36Z")

</div>

Yeah, in general, long context seems to be a major direction, reinforced by better efficiency and of course new architectures. Medium to long term, I’m also expecting potentially a form of specialization and a need for orchestration. Honestly, in the short term, I would probably focus on infrastructure, on both the software and hardware sides. So, training Trinity Large TrueBase or Base looks like probably the safest option now. And development of Julia native model(s).

---

<div class="post-metadata">

**Author:** ![Myar\_joyia](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/myar_joyia/32/223456_2.png) [@Myar\_joyia](https://discourse.julialang.org/u/Myar_joyia)\
**Post date:** [August 22, 2026, 5:21pm UTC](https://discourse.julialang.org/t/proof-of-concept-llm-chatbot-built-with-julia-keemenalm-jl/137961/44 "2026-08-22T17:21:58Z")

</div>

I agree that focusing on infrastructure first seems like a very practical direction. Beyond training a larger base model, I think it would be interesting to explore what a Julia-native stack for model development and inference could look like in practice. For example, having Julia-native tooling for efficient inference, context management, recursive workflows, and benchmarking could make it much easier to experiment with these newer architectures. It might also be useful to define a few concrete benchmarks so we can compare different approaches and see where Julia can offer real advantages rather than simply reproducing existing workflows.

---

<div class="post-metadata">

**Author:** ![j\_u](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/j_u/32/219081_2.png) [@j\_u](https://discourse.julialang.org/u/j_u)\
**Post date:** [August 22, 2026, 6:37pm UTC](https://discourse.julialang.org/t/proof-of-concept-llm-chatbot-built-with-julia-keemenalm-jl/137961/45 "2026-08-22T18:37:32Z")

</div>

With full respect, I don’t believe anyone intended to reproduce existing workflows, especially in the long term. And as far as my current understanding goes, Julia already seems highly competitive in terms of inference, at least for smaller models. But at least one important question remains open: what’s the legal framework?

---

<div class="post-metadata">

**Author:** ![mantzaris](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mantzaris/32/3852_2.png) [@mantzaris](https://discourse.julialang.org/u/mantzaris)\
**Post date:** [August 22, 2026, 6:38pm UTC](https://discourse.julialang.org/t/proof-of-concept-llm-chatbot-built-with-julia-keemenalm-jl/137961/46 "2026-08-22T18:38:27Z")

</div>

> [@Myar\_joyia](#):
>
> ed to where RLMs are heading. I’d also be curious to see whether some of these ideas could be practically explored in Julia, particul

I have tried to work with other recursive architectures in the past to be disappointed in the end and finding it a challenge. Maybe it was a bad experience.

---

<div class="post-metadata">

**Author:** ![mantzaris](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mantzaris/32/3852_2.png) [@mantzaris](https://discourse.julialang.org/u/mantzaris)\
**Post date:** [August 22, 2026, 6:43pm UTC](https://discourse.julialang.org/t/proof-of-concept-llm-chatbot-built-with-julia-keemenalm-jl/137961/47 "2026-08-22T18:43:59Z")

</div>

Yes, working on tooling would be great, but I think at least another more more robust and probably necessarily larger model might be needed first to see the fruits of the tooling. I am thinking about making a 1B-3B model, with much more data and then maybe work on tooling. But reading the posts from Palli, I think that fast attention and quantization might be needed actually before. Tell me what you all think of this roadmap:

Fast attention \> quantization \> Train 1B model on more data \> tooling

It would be a great help if some of the more academically inclined here want to join in and write some papers together during the process as well as plan the concrete next steps.

Let me know your thoughts

---

<div class="post-metadata">

**Author:** ![j\_u](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/j_u/32/219081_2.png) [@j\_u](https://discourse.julialang.org/u/j_u)\
**Post date:** [August 22, 2026, 10:40pm UTC](https://discourse.julialang.org/t/proof-of-concept-llm-chatbot-built-with-julia-keemenalm-jl/137961/48 "2026-08-22T22:40:18Z")

</div>

I’m afraid I don’t have much to add to the discussion at this moment. It’s been a pleasure. Thanks a lot. Wishing you the best.

---

<div class="post-metadata">

**Author:** ![Myar\_joyia](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/myar_joyia/32/223456_2.png) [@Myar\_joyia](https://discourse.julialang.org/u/Myar_joyia)\
**Post date:** [August 23, 2026, 4:20pm UTC](https://discourse.julialang.org/t/proof-of-concept-llm-chatbot-built-with-julia-keemenalm-jl/137961/49 "2026-08-23T16:20:32Z")

</div>

Thanks, j\_u. I really enjoyed the discussion as well. I think there’s still a lot worth exploring here, especially around how Julia could support these newer model architectures in a more native way.

I’ll keep thinking about the roadmap and the practical challenges around fast attention, quantization, and scaling the model. Hopefully we can turn some of these ideas into concrete experiments and see what works in practice.

Thanks again for sharing your thoughts, and I hope we can continue the discussion sometime in the future.

---

<div class="post-metadata">

**Author:** ![j\_u](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/j_u/32/219081_2.png) [@j\_u](https://discourse.julialang.org/u/j_u)\
**Post date:** [August 23, 2026, 9:15pm UTC](https://discourse.julialang.org/t/proof-of-concept-llm-chatbot-built-with-julia-keemenalm-jl/137961/50 "2026-08-23T21:15:08Z")

</div>

Absolutely, it seems like this space is developing rapidly. I’m always happy to make friends and exchange ideas. Again, it’s been a pleasure to have the opportunity to participate in this discussion.

---

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [August 24, 2026, 1:12am UTC](https://discourse.julialang.org/t/proof-of-concept-llm-chatbot-built-with-julia-keemenalm-jl/137961/51 "2026-08-24T01:12:55Z")

</div>

> [@mantzaris](#):
>
> this roadmap:
> 
> Fast attention \> quantization \> Train 1B model on more data \> tooling

It’s sensible, you mean FlashAttention. Then maybe a larger model, and let others do quantization? What do you mean by tooling?

I think training data, and what comes after pretraining, i.e. finetuning is very important. See my forked post. It can be done separately, and all such work reused here. I see over 50% of training data is on code, in a recent Microsoft model.

---

<div class="post-metadata">

**Author:** ![Myar\_joyia](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/myar_joyia/32/223456_2.png) [@Myar\_joyia](https://discourse.julialang.org/u/Myar_joyia)\
**Post date:** [August 24, 2026, 3:44pm UTC](https://discourse.julialang.org/t/proof-of-concept-llm-chatbot-built-with-julia-keemenalm-jl/137961/52 "2026-08-24T15:44:07Z")

</div>

I really like the direction of this project. Getting a small GPT-style model running end-to-end in Julia, including training, tokenization, artifacts, and the REPL workflow, is a useful proof of concept.

The offline side of software is also an interesting area to explore, especially for tools that need to remain usable without a constant internet connection. I’ve been looking into the “Offline Mode” concept as well, and I’ve documented some related ideas here:

It would be interesting to see where KeemenaLM.jl goes next, particularly around local/offline inference, model optimization, and making the workflow easier for Julia users.

[Previous page](https://discourse.julialang.org/t/proof-of-concept-llm-chatbot-built-with-julia-keemenalm-jl/137961.md?page=2)