I would think Julia could very well work, like for RelationalAI, one of the main Julia companies. I note it doesn’t need to be either or, Julia could be your main language, but you can call C++ code. It’s very easy to call Python, C, R, Rust etc. to and from, and from C++. It’s a bit hard to call C++ from any language, e.g. because of its name mangling, but a solved problem. I think Julia does about as well as from other languages, as other languages do, e.g. from Python.
I googled a bit to see what’s out there:
[videos]
Molham Aref and Nathan Daly describe their experience using Julia to build a next-generation knowledge graph database that combines reasoning and learning to solve problems that have historically been intractable. They explain how Julia’s unique features enabled them to build a high-performance database with less time and effort.
I bolded database there, and note it’s proprietary software, but their reasons for preferring Julia may apply to you, and possibly you could use their system…
I don’t know what’s available in C++, just showing first/very old hit I got:
LLMs are much more used from Python still, implemented by C++, but you can reuse all of that from Julia, at least in theory, e.g. people adding such to Transformers.jl, so in practice too. I don’t hear users using C++ much directly, for LLMs.
You can easily call all (LLM) models, using PythonCall.jl, see also @Tomas_Pevny’s about making it more user-friendly, integrating into Transformers.jl (most models are not there)