Julia vs C++ for Automated Reasoning

[Rajaram_Venkataraman]

Hello - I have to develop an automated reasoning system that combines the power of f language models and graph processing rather than HoQ, Isabelle or LEAN type approach. It requires new development and plan to use either C++ or Julia. Could someone please help by letting me know if Julia has the required packages? I need the following packages

1. LaTeX Processing
2. Knowledge Graph
3. LLM
4. GNNs (Graph Neural Networks)

[mrufsvold]

1. Latexify.jl
2. Not sure what you’re asking for here. If you want a fully fledged database, there is TypeDBClient.jl, which is a client for an TypeDB. If you just want to build graphs in Julia, then check out Graphs.jl
3. Transformers.jl
4. GraphNeuralNetworks.jl

[Palli]

@Rajaram_Venkataraman

plan to use either C++ or Julia.

I would think Julia could very well work, like for RelationalAI, one of the main Julia companies. I note it doesn’t need to be either or, Julia could be your main language, but you can call C++ code. It’s very easy to call Python, C, R, Rust etc. to and from, and from C++. It’s a bit hard to call C++ from any language, e.g. because of its name mangling, but a solved problem. I think Julia does about as well as from other languages, as other languages do, e.g. from Python.

I googled a bit to see what’s out there:

Building a RKGS in Julia - RelationalAI

[videos]
Molham Aref and Nathan Daly describe their experience using Julia to build a next-generation knowledge graph database that combines reasoning and learning to solve problems that have historically been intractable. They explain how Julia’s unique features enabled them to build a high-performance database with less time and effort.

I bolded database there, and note it’s proprietary software, but their reasons for preferring Julia may apply to you, and possibly you could use their system…

I don’t know what’s available in C++, just showing first/very old hit I got:

GitHub - kckishan/knowledgegraph: Knowlegde Graph Implementation in C++

LLMs are much more used from Python still, implemented by C++, but you can reuse all of that from Julia, at least in theory, e.g. people adding such to Transformers.jl, so in practice too. I don’t hear users using C++ much directly, for LLMs.

You can easily call all (LLM) models, using PythonCall.jl, see also @Tomas_Pevny’s about making it more user-friendly, integrating into Transformers.jl (most models are not there)

A discrepancy in self-attention between python and Julia (Transformers)

Hi, I am continuing the porting of microsoft Phi model to the Transformers.jl. The most complicated is the self-attention, where I am lost, but gradually removing abstraction layers of Neuralattentionlib.jl to get to the gist. Right now, I am stucked with a following problem of computing weights in self-attention. In Julia, I have julia> size(query_rot_states) (64, 32, 6) julia> size(key_rot_states) (64, 32, 6) julia> attn_weights = scaled_dot_product_score(query_rot_states, key_rot_states)…

[Tomas_Pevny]

Hi Rajaram,

this is an interesting topic. We had a paper this year at neurips about optimizing heuristic function for planning problems, see the paper here: Optimize Planning Heuristics to Rank, not to Estimate Cost-to-Goal | OpenReview. The part that implemented heuristics for general planning problems defined by PDDL was written in Julia and we are still working on it. We originally relied on GNNs, but then we realized that predicates with higher arity might be useful and wrote a custom tooling, which does not have much documentation unfortunately. I have two Bc students working on it, but I am not sure, if it will improve the documentation. I would be happy to help, if you detail your plans.

I would go for Julia rather than C++, since you would be able to reach your goal faster.