# Is Rust (and/or Python) the new high-level API (here ANN/GGUF/LLama example)

**URL:** <https://discourse.julialang.org/t/is-rust-and-or-python-the-new-high-level-api-here-ann-gguf-llama-example/121721>\
**Category:** Offtopic\
**Created:** [October 24, 2024, 10:21pm UTC](https://discourse.julialang.org/t/is-rust-and-or-python-the-new-high-level-api-here-ann-gguf-llama-example/121721 "2024-10-24T22:21:36Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [October 24, 2024, 10:21pm UTC](https://discourse.julialang.org/t/is-rust-and-or-python-the-new-high-level-api-here-ann-gguf-llama-example/121721/1 "2024-10-24T22:21:36Z")

</div>

This state-of-the-art library has Rust and Python API, but ironically no (C++) high-level one despite written in C++:

> **[Is there high-level API for LLama.cpp on C++? · ggerganov/llama.cpp ·...](https://github.com/ggerganov/llama.cpp/discussions/8074)**
>
> I am trying to integrate LLama.cpp in a C++ project that I have and I am looking for high level APIs to interact with it, the closest thing that I got was in examples/main, however it contains a lo...

> I’m guessing C-like C++ is kinda necessary for bindings to other [languages]

[TensorFlow, also written in C++, only had officially stable Python API (while Julia’s API better until unmaintained).]

If we want to call this library then we can for sure, using PythonCall.jl, also an option to call Rust’s API. So which would you prefer?

You might think why not use:

> **[GitHub - cafaxo/Llama2.jl: Julia package for inference and training of...](https://github.com/cafaxo/Llama2.jl)**
>
> Julia package for inference and training of Llama-style language models

> - GGUF models: Llama 2, Llama 3, and Phi-3 (not all quantization variants may work)
> - Andrej Karpathy’s llama2.c format

Note there, it’s only the format, Karpathy’s excellent llama2.c isn’t actually used, nor would it do if used.

llama.cpp provides all the GGUF and all the quantization types, and I’m not sure there’s any real alternative.

So why is Llama[2].jl being made from scratch in _pure_ Julia? It’s great that you can, but not really needed, or even wanted? I think we should reuse great code.

I think Julia could well be the high-level API for end-users (but also for other languages, i.e. replacing C++ as implementation language).

Until then, should we rather be wrapping Rust; or Python (in general, not just for this)?

Wrapping C++ is possible but famously annoying, and doesn’t really matter if you can or not if programs (increasingly?) do not provide C++ API. Rust also has some issues (or just this one solvable?), it can rearrange structs (good for it, for performance, bad for languages wrapping Rust, then you must forbid it, which is a possibility for “C-like API”; though in most cases not need? You just wrap API, not expose structs).

---

<div class="post-metadata">

**Author:** ![svilupp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/svilupp/32/34933_2.png) [@svilupp](https://discourse.julialang.org/u/svilupp)\
**Post date:** [October 26, 2024, 9:11pm UTC](https://discourse.julialang.org/t/is-rust-and-or-python-the-new-high-level-api-here-ann-gguf-llama-example/121721/2 "2024-10-26T21:11:56Z")

</div>

Just to throw it out there — there is [GitHub - marcom/Llama.jl: Julia interface to llama.cpp, a C/C++ library for running language models](https://github.com/marcom/Llama.jl) which wraps llama.cpp. But it’s many versions behind, we should update the jll!