Diffusion-based drafting LLM breakthroughs, and maybe we should work on fine-tuning for Julia; and inference rather than pretraining?

[Palli]

I’m no expert on fine-tuning (or middle tuning). I was thinking just get a good recent model like (then fine-tune on Julia, but not with Julia tools):

Qwen/Qwen3.8-27B · Hugging Face

If not Kimi K3… or some model like ornith-ai/Ornith-1.5-35B-A3B · Hugging Face

I see some prefer Qwen3.6 to 3.8 for some things… while stating 3.8 beats for other things. Qwen-3.8-Max may be best but I don’t find it publicly… https://qwen.ai/blog?id=qwen3.8

10+ Days of Autonomous Coding: Building a Self-Evolving Harness

…

Reproduce a research paper — then improve it

…

Autonomous Chip Design and Closed-Loop Feedback-Driven Optimization

I’m really intrigued by 3.8-Max (since I’m now looking into chip design…). The new DeepSeek harness is really intriguing, and it might be best or Prime Agent (both intriguing, not sure which is better, since also both rather new).

Some old process:

experiments/cheater-7b-finetune/README.md

FYI: @cpfiffer other threads on this, maybe further discussion should continue there:

Fine-tuning an LLM for Julia, updates

Howdy. This is a follow-up to this post on a few generative AI tasks for the Julia community, namely, producing a dataset of good Julia code and fine-tuning a model for Julia specifically. There’s a few bits and pieces we’re working on now. Preparing a high-quality dataset of raw Julia code samples. Preparing a high-quality instruction dataset, which maps user requests to LLM outputs. Fine-tuning a language model specifically for Julia. The JuliaGenAI org had a meeting last week (notes from…