# Diffusion-based drafting LLM breakthroughs, and maybe we should work on fine-tuning for Julia; and inference rather than pretraining?

**URL:** <https://discourse.julialang.org/t/diffusion-based-drafting-llm-breakthroughs-and-maybe-we-should-work-on-fine-tuning-for-julia-and-inference-rather-than-pretraining/138972>\
**Category:** Offtopic\
**Created:** [August 21, 2026, 12:08pm UTC](https://discourse.julialang.org/t/diffusion-based-drafting-llm-breakthroughs-and-maybe-we-should-work-on-fine-tuning-for-julia-and-inference-rather-than-pretraining/138972 "2026-08-21T12:08:16Z")\
**Posts on this page:** 1\
**Showing post:** 3

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [August 24, 2026, 12:55am UTC](https://discourse.julialang.org/t/diffusion-based-drafting-llm-breakthroughs-and-maybe-we-should-work-on-fine-tuning-for-julia-and-inference-rather-than-pretraining/138972/3 "2026-08-24T00:55:46Z")

</div>

I’m no expert on fine-tuning (or middle tuning). I was thinking just get a good recent model like (then fine-tune on Julia, but not with Julia tools):

> **[Qwen/Qwen3.8-27B · Hugging Face](https://huggingface.co/Qwen/Qwen3.8-27B)**
>
> We’re on a journey to advance and democratize artificial intelligence through open source and open science.

If not Kimi K3… or some model like [ornith-ai/Ornith-1.5-35B-A3B · Hugging Face](https://huggingface.co/ornith-ai/Ornith-1.5-35B-A3B)

I see some prefer Qwen3.6 to 3.8 for some things… while stating 3.8 beats for other things. Qwen-3.8-Max may be best but I don’t find it publicly… [https://qwen.ai/blog?id=qwen3.8](https://qwen.ai/blog?id=qwen3.8)

> ### 10+ Days of Autonomous Coding: Building a Self-Evolving Harness
> 
> …
> 
> ### Reproduce a research paper — then improve it
> 
> …
> 
> ### Autonomous Chip Design and Closed-Loop Feedback-Driven Optimization

I’m really intrigued by 3.8-Max (since I’m now looking into chip design…). The new DeepSeek harness is really intriguing, and it might be best or Prime Agent (both intriguing, not sure which is better, since also both rather new).

Some old process:

> <https://github.com/svilupp/Julia-LLM-Leaderboard/blob/main/experiments/cheater-7b-finetune/README.md>

FYI: @cpfiffer other threads on this, maybe further discussion should continue there:

> [@Fine-tuning an LLM for Julia, updates](https://discourse.julialang.org/t/fine-tuning-an-llm-for-julia-updates/114564):
>
> Howdy. This is a follow-up to [this post](https://discourse.julialang.org/t/an-llm-fine-tuned-for-julia-call-for-comments-help/113462) on a few generative AI tasks for the Julia community, namely, producing a dataset of good Julia code and fine-tuning a model for Julia specifically. There’s a few bits and pieces we’re working on now. Preparing a high-quality dataset of raw Julia code samples. Preparing a high-quality instruction dataset, which maps user requests to LLM outputs. Fine-tuning a language model specifically for Julia. The [JuliaGenAI org](https://juliagenai.org/) had a meeting last week ([notes](https://docs.google.com/document/d/1ViYSlg6KVqjJ7YWyRFe1-tJRzrtcEvtmc4M05Xlm8Xs/edit#heading=h.oqmvyhq7grkn) from…

---

_[View the full topic](https://discourse.julialang.org/t/diffusion-based-drafting-llm-breakthroughs-and-maybe-we-should-work-on-fine-tuning-for-julia-and-inference-rather-than-pretraining/138972)._