# Bring Julia code to embedded hardware (ARM)

**URL:** <https://discourse.julialang.org/t/bring-julia-code-to-embedded-hardware-arm/19979>\
**Category:** Maker\
**Created:** [January 23, 2019, 12:57pm UTC](https://discourse.julialang.org/t/bring-julia-code-to-embedded-hardware-arm/19979 "2019-01-23T12:57:50Z")\
**Posts on this page:** 13\
**Page:** 5

<div class="post-metadata">

**Author:** ![cirobr](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cirobr/32/219994_2.png) [@cirobr](https://discourse.julialang.org/u/cirobr)\
**Post date:** [June 11, 2023, 1:57pm UTC](https://discourse.julialang.org/t/bring-julia-code-to-embedded-hardware-arm/19979/81 "2023-06-11T13:57:27Z")

</div>

UPDATE

I’ve reached to the conclusion that Flux is no longer usable if the system does not have GPU. Given that Flux is strongly dependent on CUDA, that means ML apps using Flux on IoT devices such as Raspberry Pi are no longer viable. Newest CUDA release apparently blocks that (till recently it was enabled).

Using an AArch64 device, clean Ubuntu 22.04 install, clean CUDA toolkit install, and clean Julia 1.9.1 install. Pkg.add(“CUDA”) works. Using CUDA fails:

```julia
julia> using CUDA
┌ Error: Failed to initialize CUDA
│ exception =
│ CUDA error (code 100, CUDA_ERROR_NO_DEVICE)

```

Thus, Jetson Nano is the only game…

---

<div class="post-metadata">

**Author:** ![ToucheSir](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/touchesir/32/14411_2.png) [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)\
**Post date:** [June 11, 2023, 2:03pm UTC](https://discourse.julialang.org/t/bring-julia-code-to-embedded-hardware-arm/19979/82 "2023-06-11T14:03:49Z")

</div>

Did you actually open an issue on the CUDA.jl side as recommended on GitHub? `using CUDA` has intentionally been designed to be a no-op on platforms which don’t have a driver installed, so I’m pretty sure this counts as a bug.

---

<div class="post-metadata">

**Author:** ![Oscar\_Smith](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/oscar_smith/32/25343_2.png) [@Oscar\_Smith](https://discourse.julialang.org/u/Oscar_Smith)\
**Post date:** [June 11, 2023, 3:04pm UTC](https://discourse.julialang.org/t/bring-julia-code-to-embedded-hardware-arm/19979/83 "2023-06-11T15:04:13Z")

</div>

To me it seems like it probably would be a better idea for Flux to make the entire GPU stack a weak dep now that we have those in 1.9. There’s no reason to depend on all the GPU stuff for users that don’t have GPUs.

---

<div class="post-metadata">

**Author:** ![ToucheSir](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/touchesir/32/14411_2.png) [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)\
**Post date:** [June 11, 2023, 4:06pm UTC](https://discourse.julialang.org/t/bring-julia-code-to-embedded-hardware-arm/19979/84 "2023-06-11T16:06:13Z")

</div>

Easier said than done unfortunately. We’ve run into a number of roadblocks trying to make package extensions work while maintaining backwards compat (Flux supports 1.6 and I see people on 1.7 worryingly often) without creating backport branches on a bunch of repos (which did not work out well the last time it was tried). The latest one we ran into is [make NNlibCUDA an extension by CarloLucibello · Pull Request #492 · FluxML/NNlib.jl · GitHub](https://github.com/FluxML/NNlib.jl/pull/492#discussion_r1194909045). Given I’ve already read complaints about import word salad when it comes to using FluxML packages, you can see why these blockers are non-trivial to resolve.

---

<div class="post-metadata">

**Author:** ![cirobr](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cirobr/32/219994_2.png) [@cirobr](https://discourse.julialang.org/u/cirobr)\
**Post date:** [June 12, 2023, 1:01pm UTC](https://discourse.julialang.org/t/bring-julia-code-to-embedded-hardware-arm/19979/85 "2023-06-12T13:01:30Z")

</div>

As a researcher, I’m open to discuss on how to collaborate on bringing a solution for joining Flux and embedded Julia. This is a topic of great interest. If the solution involves Flux 2.0 with less CUDA dependency, so be it.

As a product developer that planned to use embedded Julia in a product, I’m afraid this path is no longer possible. One option in sight would be Python/TinyML - a number of chip suppliers already made it work even on Cortex-M family.

---

<div class="post-metadata">

**Author:** ![cirobr](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cirobr/32/219994_2.png) [@cirobr](https://discourse.julialang.org/u/cirobr)\
**Post date:** [June 12, 2023, 2:12pm UTC](https://discourse.julialang.org/t/bring-julia-code-to-embedded-hardware-arm/19979/86 "2023-06-12T14:12:05Z")

</div>

Just did it, and the immediate solution was … not using CUDA!

[https://github.com/JuliaGPU/CUDA.jl/issues/1952#](https://github.com/JuliaGPU/CUDA.jl/issues/1952#)

---

<div class="post-metadata">

**Author:** ![ToucheSir](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/touchesir/32/14411_2.png) [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)\
**Post date:** [June 12, 2023, 2:59pm UTC](https://discourse.julialang.org/t/bring-julia-code-to-embedded-hardware-arm/19979/87 "2023-06-12T14:59:49Z")

</div>

No, that’s not the suggestion. What appears to be happening is that CUDA.jl is detecting you have the proprietary Nvidia driver installed on your system for some reason. Since you presumably _don’t_ have a Nvidia GPU attached to said system, you’ll want to uninstall the driver and then the issue should go away. If for some reason you _must_ have the driver installed despite not having a GPU attached to the system (edit: or you don’t have it installed), I would mention that on the GitHub issue.

---

<div class="post-metadata">

**Author:** ![cirobr](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cirobr/32/219994_2.png) [@cirobr](https://discourse.julialang.org/u/cirobr)\
**Post date:** [June 12, 2023, 3:11pm UTC](https://discourse.julialang.org/t/bring-julia-code-to-embedded-hardware-arm/19979/88 "2023-06-12T15:11:37Z")

</div>

This is not my understanding. Being the man-in-the middle is not being practical at all. Perhaps if both, Flux and JuliaGPU, try to duplicate the issue as a team?

There is definitely an issue to be solved by experts.

---

<div class="post-metadata">

**Author:** ![ToucheSir](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/touchesir/32/14411_2.png) [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)\
**Post date:** [June 12, 2023, 3:29pm UTC](https://discourse.julialang.org/t/bring-julia-code-to-embedded-hardware-arm/19979/89 "2023-06-12T15:29:57Z")

</div>

> [@cirobr](#):
>
> This is not my understanding. Being the man-in-the middle is not being practical at all

I know it’s not evident from this discussion, but you’re not the man-in-the-middle here. Tim and I have seen this issue many times already, hence why we’re both asking if you can test a couple more things.

> [@cirobr](#):
>
> Perhaps if both, Flux and JuliaGPU, try to duplicate the issue as a team?

The problem is that we need an environment like yours to replicate the issue! If you know the exact specifications of the machine you’re working on you could share them here and hope someone else has a similar one to test, but otherwise I’d recommend trying to answer the following questions from my post above:

- Do you have nvidia drivers installed on this machine?
- If so, can you uninstall those drivers? Does uninstalling make the error go away?

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [June 12, 2023, 7:04pm UTC](https://discourse.julialang.org/t/bring-julia-code-to-embedded-hardware-arm/19979/90 "2023-06-12T19:04:24Z")

</div>

> [@ToucheSir](#):
>
> - Do you have nvidia drivers installed on this machine?
> - If so, can you uninstall those drivers? Does uninstalling make the error go away?

I added some details on the issue, but essentially, you’ll want to figure out which package on your system provides `libcuda.so`. Only removing what provides `nvidia.ko`, aka. the kernel-level driver, is not sufficient. You also need to remove the user-space driver.

But once more, the output you showed in the issue is just an `@error` message. It should not break Flux. If it does, then please provide more details (what broke? can you provide a backtrace?) so that we can help you better.

---

<div class="post-metadata">

**Author:** ![minetest2048](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/minetest2048/32/45961_2.png) [@minetest2048](https://discourse.julialang.org/u/minetest2048)\
**Post date:** [June 13, 2023, 3:01am UTC](https://discourse.julialang.org/t/bring-julia-code-to-embedded-hardware-arm/19979/91 "2023-06-13T03:01:12Z")

</div>

This is where hopefully package extensions/ weakdeps get adopted more. I did have a related problem where JuliaGNSS depends on CUDA.jl just to check whether the hw have Nvidia GPU and suggest enabling GPU acceleration, while I’m running it on Raspberry Pi.

Another thing is [Enable JITLink in aarch64 linux. by giordano · Pull Request #49745 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/pull/49745) is very close to be merged, which fixes a lot of segfaults on at least Rock Pi 4 and QEMU.  
With Julia can now run on QEMU then cross-compilation (ish) can be done, still QEMU overhead in translating ARM to x86 is still too high

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [June 14, 2023, 7:39am UTC](https://discourse.julialang.org/t/bring-julia-code-to-embedded-hardware-arm/19979/92 "2023-06-14T07:39:11Z")

</div>

> [@minetest2048](#):
>
> I did have a related problem where JuliaGNSS depends on CUDA.jl just to check whether the hw have Nvidia GPU and suggest enabling GPU acceleration, while I’m running it on Raspberry Pi.

Yeah, we will probably need a tiny packages to detect the availability of GPU software and hardware so that people can check whether CUDA.jl is likely to work without having to depend on all of CUDA.jl.

Anyway, this is pretty off-topic to what was originally being discussed here.

---

<div class="post-metadata">

**Author:** ![cirobr](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cirobr/32/219994_2.png) [@cirobr](https://discourse.julialang.org/u/cirobr)\
**Post date:** [June 14, 2023, 4:38pm UTC](https://discourse.julialang.org/t/bring-julia-code-to-embedded-hardware-arm/19979/93 "2023-06-14T16:38:31Z")

</div>

Until the issue is fixed, have downgraded CUDA Pkg to v3.13.1. Development and compilation remain on a large AArch64/no GPU instance, then rsync to the Raspberry Pi.

[Previous page](https://discourse.julialang.org/t/bring-julia-code-to-embedded-hardware-arm/19979.md?page=4)