# Is there a C/C++/Rust/Go library that can do inference on Flux models?

**URL:** <https://discourse.julialang.org/t/is-there-a-c-c-rust-go-library-that-can-do-inference-on-flux-models/102581>\
**Category:** Machine Learning\
**Tags:** question, flux, c\
**Created:** [August 8, 2023, 12:56am UTC](https://discourse.julialang.org/t/is-there-a-c-c-rust-go-library-that-can-do-inference-on-flux-models/102581 "2023-08-08T00:56:06Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![tchebycheff](https://avatars.discourse-cdn.com/v4/letter/t/779978/32.png) [@tchebycheff](https://discourse.julialang.org/u/tchebycheff)\
**Post date:** [August 8, 2023, 12:56am UTC](https://discourse.julialang.org/t/is-there-a-c-c-rust-go-library-that-can-do-inference-on-flux-models/102581/1 "2023-08-08T00:56:06Z")

</div>

If not, is it useful to have one at least for simple layers like Dense, Conv, Scale, etc.?

---

<div class="post-metadata">

**Author:** ![xgdgsc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xgdgsc/32/608_2.png) [@xgdgsc](https://discourse.julialang.org/u/xgdgsc)\
**Post date:** [August 8, 2023, 2:21am UTC](https://discourse.julialang.org/t/is-there-a-c-c-rust-go-library-that-can-do-inference-on-flux-models/102581/2 "2023-08-08T02:21:59Z")

</div>

Would building a C lib with PackageCompiler work?

---

<div class="post-metadata">

**Author:** ![tchebycheff](https://avatars.discourse-cdn.com/v4/letter/t/779978/32.png) [@tchebycheff](https://discourse.julialang.org/u/tchebycheff)\
**Post date:** [August 8, 2023, 2:40am UTC](https://discourse.julialang.org/t/is-there-a-c-c-rust-go-library-that-can-do-inference-on-flux-models/102581/3 "2023-08-08T02:40:29Z")

</div>

Thanks. That looks interesting. Exploring.

---

<div class="post-metadata">

**Author:** ![tchebycheff](https://avatars.discourse-cdn.com/v4/letter/t/779978/32.png) [@tchebycheff](https://discourse.julialang.org/u/tchebycheff)\
**Post date:** [August 8, 2023, 4:21am UTC](https://discourse.julialang.org/t/is-there-a-c-c-rust-go-library-that-can-do-inference-on-flux-models/102581/4 "2023-08-08T04:21:21Z")

</div>

Where can I find documentation for jl\_ functions and types used in C? Thanks.

---

<div class="post-metadata">

**Author:** ![xgdgsc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xgdgsc/32/608_2.png) [@xgdgsc](https://discourse.julialang.org/u/xgdgsc)\
**Post date:** [August 8, 2023, 4:40am UTC](https://discourse.julialang.org/t/is-there-a-c-c-rust-go-library-that-can-do-inference-on-flux-models/102581/5 "2023-08-08T04:40:59Z")

</div>

[https://docs.julialang.org/en/v1/manual/embedding/](https://docs.julialang.org/en/v1/manual/embedding/)

[https://docs.julialang.org/en/v1/base/c/](https://docs.julialang.org/en/v1/base/c/)

---

<div class="post-metadata">

**Author:** ![tchebycheff](https://avatars.discourse-cdn.com/v4/letter/t/779978/32.png) [@tchebycheff](https://discourse.julialang.org/u/tchebycheff)\
**Post date:** [August 8, 2023, 5:58am UTC](https://discourse.julialang.org/t/is-there-a-c-c-rust-go-library-that-can-do-inference-on-flux-models/102581/6 "2023-08-08T05:58:46Z")

</div>

Thank you. I did go through those sections. There are very useful examples of function usage, not really documentation. I guess I need to go through the header files.

I got a minimal example with Flux working though.

---

<div class="post-metadata">

**Author:** ![albheim](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/albheim/32/34660_2.png) [@albheim](https://discourse.julialang.org/u/albheim)\
**Post date:** [August 8, 2023, 8:08am UTC](https://discourse.julialang.org/t/is-there-a-c-c-rust-go-library-that-can-do-inference-on-flux-models/102581/7 "2023-08-08T08:08:37Z")

</div>

It seems like [ONNX.jl](https://github.com/FluxML/ONNX.jl) is in the middle of some restructuring and is not currently supporting Flux.jl, but otherwise that could possibly be one way of moving a model to another language. Not sure what the exact status is, but does not seem to have too much activity recently 😕

---

<div class="post-metadata">

**Author:** ![tchebycheff](https://avatars.discourse-cdn.com/v4/letter/t/779978/32.png) [@tchebycheff](https://discourse.julialang.org/u/tchebycheff)\
**Post date:** [August 8, 2023, 8:20am UTC](https://discourse.julialang.org/t/is-there-a-c-c-rust-go-library-that-can-do-inference-on-flux-models/102581/8 "2023-08-08T08:20:09Z")

</div>

Thank you.

---

<div class="post-metadata">

**Author:** ![ToucheSir](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/touchesir/32/14411_2.png) [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)\
**Post date:** [August 8, 2023, 1:36pm UTC](https://discourse.julialang.org/t/is-there-a-c-c-rust-go-library-that-can-do-inference-on-flux-models/102581/9 "2023-08-08T13:36:13Z")

</div>

That just means you can’t _import_ ONNX files as Flux models (check out [GitHub - DrChainsaw/ONNXNaiveNASflux.jl: Import/export ONNX models](https://github.com/DrChainsaw/ONNXNaiveNASflux.jl) if you must have that). Exporting to ONNX should work now for a decent number of models. I think the README could use some updating to reflect that, PRs welcome.

---

<div class="post-metadata">

**Author:** ![tchebycheff](https://avatars.discourse-cdn.com/v4/letter/t/779978/32.png) [@tchebycheff](https://discourse.julialang.org/u/tchebycheff)\
**Post date:** [August 9, 2023, 5:23am UTC](https://discourse.julialang.org/t/is-there-a-c-c-rust-go-library-that-can-do-inference-on-flux-models/102581/10 "2023-08-09T05:23:56Z")

</div>

Thanks. If this can work with C/C++, it would be great.  
The C-API route of running a jl environment is not ideal.

---

<div class="post-metadata">

**Author:** ![ToucheSir](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/touchesir/32/14411_2.png) [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)\
**Post date:** [August 9, 2023, 5:30am UTC](https://discourse.julialang.org/t/is-there-a-c-c-rust-go-library-that-can-do-inference-on-flux-models/102581/11 "2023-08-09T05:30:24Z")

</div>

Once you’ve exported a model to ONNX, you can load it in the runtime of your choice. The most popular one is [https://onnxruntime.ai/](https://onnxruntime.ai/), which has C/C++ bindings. I know at least a few other runtimes should as well, but not exact names. [ONNX | Supported Tools](https://onnx.ai/supported-tools.html) may be a good starting point to read further there.

---

<div class="post-metadata">

**Author:** ![tchebycheff](https://avatars.discourse-cdn.com/v4/letter/t/779978/32.png) [@tchebycheff](https://discourse.julialang.org/u/tchebycheff)\
**Post date:** [August 9, 2023, 12:08pm UTC](https://discourse.julialang.org/t/is-there-a-c-c-rust-go-library-that-can-do-inference-on-flux-models/102581/12 "2023-08-09T12:08:39Z")

</div>

Thank you.