Continuing the discussion from Anthropic gave me 6 months of Claude Max for my contributions to the Julia ecosystem :slight_smile: : [image] Anthropic gave me 6 months of Claude Max for my contributions to the Julia ecosystem slight_smile Proliferation of vibe-coded packages will make the ecosyst…

I think it is really not possible to run Kimi K3 or other really big LLMs locally without proper hardware. Look here: Release AirLLM v3.1.0 — Kimi K3 (2.8T) on a single card · lyogavin/airllm · GitHub , it is incredible, yes, that one can run it with only 4GB of VRAM, though the speed is 292 s/token,…

IMHO, given the price of Deepseek Flash v4 ($0.084 / $0.168 per 1M tokens), you will spend a fraction of what you need considering the hardware, energy, etc. to run locally an acceptable model.

I’m afraid @Tortar and @Ronis_BR are right. Unless you’re doing a massive amount of coding and running concurrent requests, it’s probably not economically justifiable to run the latest large models locally on your own hardware. Or at least this is my current understanding. NVIDIA is offering a dece…

[image] Palli: GLM 5.2 is likely one of the best, I tried it inside Claude Code and basically it is a piece of … Not to mentioned that it consumed the credits some 10x faster that even Opus. Tried also GPT “Sol”. Only slightly better than GLM. And I almost always had to launch Opus to correc…

[image] joa-quim: PS I am very jealous of Ronan’s gift (muitos parabéns). Thanks! (Obrigado!) :slight_smile:

Kimi k3 seems to do pretty well compared to opus!

[image] sdanisch: Kimi k3 seems to do pretty well compared to opus! Locally, I guess those kinds of things are only possible in Germany. :- ) But seriously, how are you running it, @sdanisch ? The vLLM requirement seems to be v0.27.0, but v0.26.0 is currently the latest official release. @Pal…

Great, you mean Kimi K3 for coding Julia? Good to know [then]. Not just for general programming, or some other specific language? And what harness, if any (I think they are critical, and not all as good) do you use with it? It’s a good thing AirLLM added: [image] GitHub - lyogavin/ai…

As for AirLLM and Mixture-of-Kittens: I’ll be honest with you, I’m not even planning to touch them. A lot of this is new to me. For example, I initially tried to run GLM 5.2, and it took me about five days to realize it’s probably not possible. The most success I had was with K 2.7, where the best I…

[image] sdanisch: Kimi k3 seems to do pretty well compared to opus! Good to know, I suppose it’s the best open weight model, because of its enormous size, so I would also want to know what is second best or by size class. And how slow Kimi K3 needs to be: For Kimi K3 from sqlite.ai, a token …

Which is the best open model for [Julia] coding? And practical, and still reasonably good

General Usage

Topic		Replies	Views
[ANN] Julia LLM Leaderboard - Help us make it more relevant for every day problems! Package Announcements announcement , ai , prompting	22	4110	April 5, 2024
AI tools to write (Julia) code (best/worse experience), e.g. ChatGPT, GPT 3.5 Offtopic	62	17352	May 14, 2024
Julia code generation LLM models? General Usage question	4	577	August 30, 2025
Proof of concept LLM chatbot built with Julia: KeemenaLM.jl Package Announcements	33	1416	August 3, 2026
LLM AI just for Julia? A proposal: Julia plus science LLM? General Usage machine-learning	4	1779	June 24, 2023

Which is the best open model for [Julia] coding? And practical, and still reasonably good

Related topics