so a few days ago a new AI model was released under the name of Jev. I signed up to the beta, because I thought it was an interesting new take on the entire market and the reason why I am posting this at all (this is not propaganda, I am merely a user).
What they offer is not a frontier model, that’s not the purpose. Instead Jev is a low-cost & highly specialized model that only does one thing: decisions. You feed it text context and it will either a) answer your yes/no question b) select from a list of options (categorize) c) score the result.
It is incredibly fast for an llm (nowhere near functional designs, but much faster than your usual model) and only costs USD 0.04 for 1 Mio input tokens - the output tokens are free.
To me this opens up many many options that were difficult to solve with LLMs in the past and something I believe also fits into the Julia world. Since I got access to it, I also wrote up a quick article with my first impressions that you can find on substack: My Thoughts on Jev, after the Private Beta - Paul Piper
Anyway, would love to hear your thoughts. But I am already going over potential applications in my head. It is defintely the right step in the right direction, as I hated the idea of being dependend on overly complex and heavy frontier models…
It’s fast, but not a large language model, LLM (i.e. autoregressive, based on RLHF), at least as usually understood, GenAI. I understand it to takes text in yes, and give only good decisions out, i.e. it’s for reliable automation. From their FAQ:
Is Jev just a smaller LLM?
Jev is neither small nor an LLM, hence being off the intelligence Pareto curve. Jev’s efficiency comes from optimizing for a different task. It’s built for structured decisions inside software, with an interface and training approach designed for that purpose. It understands language, but doesn’t generate free-form text or function as a chatbot.
I liked the talk from the founder, co-created or ChatGPT and one of the main researchers behind reinforcement learning with human feedback (RLHF), ChatGPT and other LLMs are about. He goes into it and the flaws and what he’s now did better.
I only learned of Jev yesterday from this video:
I recommend watching all of it (see does good videos), and the claim at the timestamp I point to. The founder is clearly not Sam Altman, but a researcher, and I like him and his talk, but it was clear he didn’t want to go into some details then pre-launch, so her talk may be more practically informative.
That would be funny if we circled back to DAG style agents with conditional edges and flow control, basically what we had to do before tool calling got good. But of course, open ended agnetic loops have far fewer structural guarantees about “staying on the rails” so to speak. I’d be interested in seeing how the DAG approach works with a much more capable model like I suppose Jev is supposed to be, and how far this approach can scale up, how crazy we can get with the number of states, etc.
Seems like a positive direction to me.
EDIT: models like Jev should also be compute bound instead of memory bandwidth bound. It’s much easier / cheaper to scale up compute than it is memory bandwidth for modern accelerators IIRC. So from a cost standpoint, it makes sense it would be cheaper since prefill is compute bound instead of memory bandwidth bound like decoding.
