# Support of Rockchip RK3588S NPU

**URL:** <https://discourse.julialang.org/t/support-of-rockchip-rk3588s-npu/89868>\
**Category:** Machine Learning\
**Tags:** question\
**Created:** [November 7, 2022, 10:20am UTC](https://discourse.julialang.org/t/support-of-rockchip-rk3588s-npu/89868 "2022-11-07T10:20:24Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![ufechner7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ufechner7/32/51363_2.png) [@ufechner7](https://discourse.julialang.org/u/ufechner7)\
**Post date:** [November 7, 2022, 10:20am UTC](https://discourse.julialang.org/t/support-of-rockchip-rk3588s-npu/89868/1 "2022-11-07T10:20:24Z")

</div>

A new ARM CPU is available, that is at least three times a powerful as a Raspberry Pi 4. It also has a neural network processing unit, which is supported by Python. Will it also be supported by Julia? What would be needed to support a new NPU?

“The RK3588S has a built-in NPU which provides up to 6 TOPS (tera operations per second) of neural network processing. The NPU supports mainstream deep learning frameworks, such as TensorFlow, Pytorch, MxNET and so on. The powerful RK3588S brings optimized neural network performance to various A.I. applications.” ( [https://www.khadas.com/edge2](https://www.khadas.com/edge2) )

---

<div class="post-metadata">

**Author:** ![ufechner7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ufechner7/32/51363_2.png) [@ufechner7](https://discourse.julialang.org/u/ufechner7)\
**Post date:** [November 7, 2022, 1:01pm UTC](https://discourse.julialang.org/t/support-of-rockchip-rk3588s-npu/89868/2 "2022-11-07T13:01:38Z")

</div>

Perhaps I should ask a more general question: Is any NPU supported by Julia or one of the Julia packages?

There are many of them: [https://en.wikichip.org/wiki/neural\_processor](https://en.wikichip.org/wiki/neural_processor)

---

<div class="post-metadata">

**Author:** ![contradict](https://avatars.discourse-cdn.com/v4/letter/c/ac91a4/32.png) [@contradict](https://discourse.julialang.org/u/contradict)\
**Post date:** [November 7, 2022, 5:51pm UTC](https://discourse.julialang.org/t/support-of-rockchip-rk3588s-npu/89868/3 "2022-11-07T17:51:50Z")

</div>

The support libraries are available as a binary here: [GitHub - rockchip-linux/rknpu2](https://github.com/rockchip-linux/rknpu2)

Writing wrappers should be possible since it is a C library, although I’m not sure what the interface should look like on the Julia side.

---

<div class="post-metadata">

**Author:** ![Krastanov](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/krastanov/32/6817_2.png) [@Krastanov](https://discourse.julialang.org/u/Krastanov)\
**Post date:** [November 7, 2022, 5:56pm UTC](https://discourse.julialang.org/t/support-of-rockchip-rk3588s-npu/89868/4 "2022-11-07T17:56:16Z")

</div>

The simplest solution would be to create a new array type, e.g. RockchipNPUArray, for which you make custom methods like `*(a::RochchipNPUArray, b::...) = call_to_C_library_rknpu2`. It should be relatively straightforward (because the NPU has only a small set of capabilities). An experienced developer can probably do it in under a week, if they have a computer running this chip and if julia compiled for it without hickups. The difficulty is finding someone who wants to do the work.

---

<div class="post-metadata">

**Author:** ![ufechner7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ufechner7/32/51363_2.png) [@ufechner7](https://discourse.julialang.org/u/ufechner7)\
**Post date:** [November 7, 2022, 5:57pm UTC](https://discourse.julialang.org/t/support-of-rockchip-rk3588s-npu/89868/5 "2022-11-07T17:57:04Z")

</div>

I think there are two different interfaces, one for creating a model and another one for using a trained model…

The interface does not look so complicated: [rknpu2/rknn\_api.h at master · rockchip-linux/rknpu2 · GitHub](https://github.com/rockchip-linux/rknpu2/blob/master/runtime/RK3588/Linux/librknn_api/include/rknn_api.h) But they should update their copyright info…

---

<div class="post-metadata">

**Author:** ![contradict](https://avatars.discourse-cdn.com/v4/letter/c/ac91a4/32.png) [@contradict](https://discourse.julialang.org/u/contradict)\
**Post date:** [November 7, 2022, 6:26pm UTC](https://discourse.julialang.org/t/support-of-rockchip-rk3588s-npu/89868/6 "2022-11-07T18:26:22Z")

</div>

I didn’t read that library after I found it, looking at it now it doesn’t actually seem to expose any fundamental operations, just the ability to load a model and provide input. Perhaps its not as useful as I though!

---

<div class="post-metadata">

**Author:** ![ufechner7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ufechner7/32/51363_2.png) [@ufechner7](https://discourse.julialang.org/u/ufechner7)\
**Post date:** [November 7, 2022, 6:30pm UTC](https://discourse.julialang.org/t/support-of-rockchip-rk3588s-npu/89868/7 "2022-11-07T18:30:50Z")

</div>

Well, this is a chip for embedded systems (and laptops). You do not train a model on such a machine, you just use pre-trained models… I think that’s the same with similar parts from NVIDIA or Google…  
For example: [USB Accelerator | Coral](https://coral.ai/products/accelerator/) should have a similar functionality…

---

<div class="post-metadata">

**Author:** ![ufechner7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ufechner7/32/51363_2.png) [@ufechner7](https://discourse.julialang.org/u/ufechner7)\
**Post date:** [November 7, 2022, 7:21pm UTC](https://discourse.julialang.org/t/support-of-rockchip-rk3588s-npu/89868/8 "2022-11-07T19:21:48Z")

</div>

Nice test/ demo: [IMX415 + NPU demo on ROCK 5B - ROCK 5 Series - Radxa Forum](https://forum.radxa.com/t/imx415-npu-demo-on-rock-5b/11319)

---

<div class="post-metadata">

**Author:** ![suavesito](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/suavesito/32/34386_2.png) [@suavesito](https://discourse.julialang.org/u/suavesito)\
**Post date:** [November 7, 2022, 7:41pm UTC](https://discourse.julialang.org/t/support-of-rockchip-rk3588s-npu/89868/9 "2022-11-07T19:41:51Z")

</div>

That should be straight-forward to make a wrapper for using [CBinding.jl](https://github.com/analytech-solutions/CBinding.jl).

---

<div class="post-metadata">

**Author:** ![contradict](https://avatars.discourse-cdn.com/v4/letter/c/ac91a4/32.png) [@contradict](https://discourse.julialang.org/u/contradict)\
**Post date:** [January 31, 2024, 5:37pm UTC](https://discourse.julialang.org/t/support-of-rockchip-rk3588s-npu/89868/10 "2024-01-31T17:37:45Z")

</div>

A very impressive reverse engineering effort for a similar NPU:

[https://blog.tomeuvizoso.net/](https://blog.tomeuvizoso.net/)

This is the NPU in the Khadas VIM3 and the Libre Computer Solitude.

---

<div class="post-metadata">

**Author:** ![younes](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/younes/32/208506_2.png) [@younes](https://discourse.julialang.org/u/younes)\
**Post date:** [May 21, 2024, 10:29am UTC](https://discourse.julialang.org/t/support-of-rockchip-rk3588s-npu/89868/11 "2024-05-21T10:29:46Z")

</div>

I did a bit of research and gathered this information about NPUs. Apparently, NVIDIA isn’t in the NPU business because their GPUs are far better than what these mainstream processing units could offer in terms of performance and computing.

| Vendor | NPU Library | GitHub Repo | Raw Full Link |
| --- | --- | --- | --- |
| ARM | Ethos-N Driver Stack | ARM-software/ethos-n-driver-stack | [https://github.com/ARM-software/ethos-n-driver-stack](https://github.com/ARM-software/ethos-n-driver-stack) |
| Qualcomm | AI Hub Models | quic/ai-hub-models | [https://github.com/quic/ai-hub-models](https://github.com/quic/ai-hub-models) |
| NXP | Arm NN i.MX Machine Learning | nxp-imx/armnn-imx | [https://github.com/nxp-imx/armnn-imx](https://github.com/nxp-imx/armnn-imx) |
| NXP | Ethos-U Driver Stack i.MX | nxp-imx/ethos-u-driver-stack-imx | [https://github.com/nxp-imx/ethos-u-driver-stack-imx](https://github.com/nxp-imx/ethos-u-driver-stack-imx) |
| Intel | Intel NPU Acceleration Library | intel/intel-npu-acceleration-library | [https://github.com/intel/intel-npu-acceleration-library](https://github.com/intel/intel-npu-acceleration-library) |
| Intel | NPU Neural Network Cost Model | intel/npu-nn-cost-model | [https://github.com/intel/npu-nn-cost-model](https://github.com/intel/npu-nn-cost-model) |
| Intel | NPU Plugin for LLVM | intel/npu-plugin-llvm | [https://github.com/intel/npu-plugin-llvm](https://github.com/intel/npu-plugin-llvm) |
| Rockchip | ezrknpu | Pelochus/ezrknpu | [https://github.com/Pelochus/ezrknpu](https://github.com/Pelochus/ezrknpu) |

---

<div class="post-metadata">

**Author:** ![younes](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/younes/32/208506_2.png) [@younes](https://discourse.julialang.org/u/younes)\
**Post date:** [May 21, 2024, 10:32am UTC](https://discourse.julialang.org/t/support-of-rockchip-rk3588s-npu/89868/12 "2024-05-21T10:32:20Z")

</div>

I guess that Julia isn’t supported because Julia is mostly used for advanced computing, where GPUs are required for performance. Therefore, is it _correct_ to assume that (most of) the Julia community isn’t “_interested_” in NPUs?