# Flux with AMD GPU(s)?

**URL:** <https://discourse.julialang.org/t/flux-with-amd-gpu-s/85218>\
**Category:** Machine Learning\
**Tags:** flux, amdgpu\
**Created:** [August 3, 2022, 8:12am UTC](https://discourse.julialang.org/t/flux-with-amd-gpu-s/85218 "2022-08-03T08:12:10Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![johnbb](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnbb/32/34233_2.png) [@johnbb](https://discourse.julialang.org/u/johnbb)\
**Post date:** [August 3, 2022, 8:12am UTC](https://discourse.julialang.org/t/flux-with-amd-gpu-s/85218/1 "2022-08-03T08:12:10Z")

</div>

Has anyone used Flux with an AMD GPU? I will be involved in a project that likely will be using the [LUMI](http://www.lumi-supercomputer.eu/) supercomputer in Finland which is based on AMD.

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [August 3, 2022, 9:53am UTC](https://discourse.julialang.org/t/flux-with-amd-gpu-s/85218/2 "2022-08-03T09:53:59Z")

</div>

Not sure about Flux in particular, but AMD GPU support has been making good progress recently as far as I understand, see:

> **[GitHub - JuliaGPU/AMDGPU.jl: AMD GPU (ROCm) programming in Julia](https://github.com/JuliaGPU/AMDGPU.jl)**
>
> AMD GPU (ROCm) programming in Julia. Contribute to JuliaGPU/AMDGPU.jl development by creating an account on GitHub.

@jpsamaroo can probably be more specific.

---

<div class="post-metadata">

**Author:** ![jpsamaroo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jpsamaroo/32/46804_2.png) [@jpsamaroo](https://discourse.julialang.org/u/jpsamaroo)\
**Post date:** [August 8, 2022, 4:20pm UTC](https://discourse.julialang.org/t/flux-with-amd-gpu-s/85218/3 "2022-08-08T16:20:29Z")

</div>

Hi @johnbb ! Flux should work with AMDGPU.jl, although many features (like CNNs or softmax) don’t work yet because we haven’t hooked up the necessary functions from ROCm’s MIOpen library. That should be pretty easy to wire up, though, so if you want to take this on, please let me know!

---

<div class="post-metadata">

**Author:** ![carstenbauer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/carstenbauer/32/4981_2.png) [@carstenbauer](https://discourse.julialang.org/u/carstenbauer)\
**Post date:** [August 8, 2022, 4:56pm UTC](https://discourse.julialang.org/t/flux-with-amd-gpu-s/85218/4 "2022-08-08T16:56:15Z")

</div>

Pinging @luraess because he’s actively working on Julia + AMD GPUs and also is testing on LUMI if I’m not mistaken.

---

<div class="post-metadata">

**Author:** ![claforte](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/claforte/32/19115_2.png) [@claforte](https://discourse.julialang.org/u/claforte)\
**Post date:** [August 8, 2022, 9:49pm UTC](https://discourse.julialang.org/t/flux-with-amd-gpu-s/85218/5 "2022-08-08T21:49:08Z")

</div>

Julian, do you maybe have a Github issue with a short list of what steps you think are required… beyond just trying each Flux test/example and seeing what’s missing/broken, then examining the CUDA.jl equivalent? Thanks!

---

<div class="post-metadata">

**Author:** ![luraess](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/luraess/32/16189_2.png) [@luraess](https://discourse.julialang.org/u/luraess)\
**Post date:** [August 8, 2022, 10:49pm UTC](https://discourse.julialang.org/t/flux-with-amd-gpu-s/85218/6 "2022-08-08T22:49:54Z")

</div>

Indeed, doing some early access tests with AMDGPU.jl, MPI.jl and ImplicitGlobalGrid.jl on LUMI. The ROCm stack is functional and accessible from AMDGPU. Currently testing with Julia v1.8.0-rc3. _“Classical” HPC though so nothing done with Flux (yet)._

---

<div class="post-metadata">

**Author:** ![darsnack](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/darsnack/32/10144_2.png) [@darsnack](https://discourse.julialang.org/u/darsnack)\
**Post date:** [August 9, 2022, 2:54am UTC](https://discourse.julialang.org/t/flux-with-amd-gpu-s/85218/7 "2022-08-09T02:54:19Z")

</div>

In addition to AMDGPU.jl providing the necessary bindings, we’ll want to create an AMD equivalent for [NNlib](https://github.com/FluxML/NNlibCUDA.jl). Once that’s in place, Flux models should just work™. Maybe we ought to create this repo and used the NNlib interface as “the list” to track all the missing pieces top down?

---

<div class="post-metadata">

**Author:** ![claforte](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/claforte/32/19115_2.png) [@claforte](https://discourse.julialang.org/u/claforte)\
**Post date:** [August 9, 2022, 6:17pm UTC](https://discourse.julialang.org/t/flux-with-amd-gpu-s/85218/8 "2022-08-09T18:17:38Z")

</div>

BTW while our 3D+ML team at AMD is using Julia and AMDGPU.jl, we’re not heavy users of Flux and NNlib yet… we write our ML kernels primarily using `KernelAbstractions.jl`. So while I wish we could address this Flux+AMDGPU limitation, we can’t prioritize it right now. But our team will be hiring 3 more research engineers soon. If anyone who reads this is interested in joining our team and supporting this use case, please message me. 🙂

---

<div class="post-metadata">

**Author:** ![jpsamaroo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jpsamaroo/32/46804_2.png) [@jpsamaroo](https://discourse.julialang.org/u/jpsamaroo)\
**Post date:** [August 9, 2022, 8:03pm UTC](https://discourse.julialang.org/t/flux-with-amd-gpu-s/85218/9 "2022-08-09T20:03:30Z")

</div>

Using the NNlib interface as our list of missing features sounds good; I’m not actively focusing on Flux-based ML right now, though, so I’ll let one of you create NNlibAMDGPU.jl (or NNlibROCm.jl, etc.). Feel free to also add comments to [Implement Neural Network primitives · Issue #11 · JuliaGPU/AMDGPU.jl · GitHub](https://github.com/JuliaGPU/AMDGPU.jl/issues/11).

---

<div class="post-metadata">

**Author:** ![johnbb](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnbb/32/34233_2.png) [@johnbb](https://discourse.julialang.org/u/johnbb)\
**Post date:** [August 18, 2022, 12:59pm UTC](https://discourse.julialang.org/t/flux-with-amd-gpu-s/85218/10 "2022-08-18T12:59:15Z")

</div>

Thanks for the response everyone. I likely don’t have the skills to contribute to an AMD NNlib, unfortunately, apart from making tests and being a keen user (through Flux). As it seems now, I will not have access to LUMI/AMD GPUs before well into 2023. It would be great if we somehow could use Flux (or similar) on AMD GPUs within the next year or two.

---

<div class="post-metadata">

**Author:** ![davide445](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/davide445/32/33681_2.png) [@davide445](https://discourse.julialang.org/u/davide445)\
**Post date:** [August 18, 2022, 4:47pm UTC](https://discourse.julialang.org/t/flux-with-amd-gpu-s/85218/11 "2022-08-18T16:47:48Z")

</div>

Due current status, will make sense starting working on Flux with CUDA and next switching to AMD as soon things got more complete?  
How much changes will be needed (of course limiting to ROCm supported components) porting the code from NV to AMD backend (the idea is using NV as primary and just cross test on AMD from time to time to know when is ready for a switch).

---

<div class="post-metadata">

**Author:** ![johnbb](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/johnbb/32/34233_2.png) [@johnbb](https://discourse.julialang.org/u/johnbb)\
**Post date:** [August 19, 2022, 7:50am UTC](https://discourse.julialang.org/t/flux-with-amd-gpu-s/85218/12 "2022-08-19T07:50:10Z")

</div>

Yes, in my case, I already have working models in Flux with single NVIDIA GPUs. In the aforementioned project, a (sub)project under the European [Destination Earth](https://digital-strategy.ec.europa.eu/en/policies/destination-earth) programme, I have/had no influence on the choice of HPC/compute resources as my task is relatively a minor one. Besides, I am possibly the only one using Julia, but of course eager to demonstrate that Julia/Flux is a viable alternative to TensorFlow and PyTorch, in particular since I don’t know Python. I guess I will mostly do the development and testing on my own computer (as often is the case), but in the end I need to have code running on AMD hardware.

---

<div class="post-metadata">

**Author:** ![davide445](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/davide445/32/33681_2.png) [@davide445](https://discourse.julialang.org/u/davide445)\
**Post date:** [August 19, 2022, 9:28am UTC](https://discourse.julialang.org/t/flux-with-amd-gpu-s/85218/13 "2022-08-19T09:28:55Z")

</div>

I will do the same.  
Concerned if there is any tool to check the Julia code and hilight ROCm unsupported features.  
Something like HIPify.

---

<div class="post-metadata">

**Author:** ![jpsamaroo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jpsamaroo/32/46804_2.png) [@jpsamaroo](https://discourse.julialang.org/u/jpsamaroo)\
**Post date:** [August 21, 2022, 1:43pm UTC](https://discourse.julialang.org/t/flux-with-amd-gpu-s/85218/14 "2022-08-21T13:43:13Z")

</div>

Unsupported features on the device side are usually reported as an error by GPUCompiler during compilation. There’s also some work going on to integrate JET with GPUCompiler to get a better idea of why code fails to compile, but this has some issues in upstream Julia that needs resolving first.

---

<div class="post-metadata">

**Author:** ![davide445](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/davide445/32/33681_2.png) [@davide445](https://discourse.julialang.org/u/davide445)\
**Post date:** [December 11, 2022, 8:11pm UTC](https://discourse.julialang.org/t/flux-with-amd-gpu-s/85218/15 "2022-12-11T20:11:53Z")

</div>

Returning in this topic, I have the choice to get an RX 6700 XT next week, my only goal being testing Julia ML related topics on AMD hw, before testing on Instinct hardware.  
Something is changed on this side?  
@claforte @luraess @pxl-th my I kindly ask if did your team made any progress regarding NNlib/Flux usage? If not, there is any plan for this?

---

<div class="post-metadata">

**Author:** ![pxl-th](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pxl-th/32/31939_2.png) [@pxl-th](https://discourse.julialang.org/u/pxl-th)\
**Post date:** [December 12, 2022, 9:38am UTC](https://discourse.julialang.org/t/flux-with-amd-gpu-s/85218/16 "2022-12-12T09:38:08Z")

</div>

Hi. There’s been some progress, specifically:

- MIOpen was added to AMDGPU.jl: [https://github.com/JuliaGPU/AMDGPU.jl/pull/320](https://github.com/JuliaGPU/AMDGPU.jl/pull/320)
- Some initial work on NNlibROC.jl: [GitHub - JuliaNeuralGraphics/NNlibROC.jl](https://github.com/JuliaNeuralGraphics/NNlibROC.jl)  
Currently it covers convolutions and batched matrix multiplications.

So for some initial support, only thing that is left to be done is make changes to Flux.jl to be able to select what backends to use.

Other things still need work.

---

<div class="post-metadata">

**Author:** ![davide445](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/davide445/32/33681_2.png) [@davide445](https://discourse.julialang.org/u/davide445)\
**Post date:** [December 12, 2022, 11:10am UTC](https://discourse.julialang.org/t/flux-with-amd-gpu-s/85218/17 "2022-12-12T11:10:53Z")

</div>

Maybe @jpsamaroo @dhairyagandhi96 know about planning regardind this?  
Was looking at [https://github.com/FluxML/Flux.jl/pull/1566](https://github.com/FluxML/Flux.jl/pull/1566)

---

<div class="post-metadata">

**Author:** ![RoyiAvital](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/royiavital/32/571_2.png) [@RoyiAvital](https://discourse.julialang.org/u/RoyiAvital)\
**Post date:** [December 12, 2022, 12:31pm UTC](https://discourse.julialang.org/t/flux-with-amd-gpu-s/85218/18 "2022-12-12T12:31:24Z")

</div>

We all know the support for ML and GPGPU for `AMD` based GPU’s is much weaker than `NVIDIA` and `CUDA`.

Might it be an opportunity for Julia?  
Namely, what if Julia will be the frontline of supporting non NVIDIA GPU’s for ML?  
AMD and Intel GPU’s are much cheaper, more memory and with less artificial disabilities.  
So promoting this niche might create an opportunity to have a hold of the market.

It requires more synched effort of development to make it work.  
On Linux there is `ROCm` and `OneAPI` and for Windows one might use `DirectML`.

Many people will be happy to drop NVIDIA, so it might generate a momentum. It is a risk and an opportunity.

---

<div class="post-metadata">

**Author:** ![davide445](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/davide445/32/33681_2.png) [@davide445](https://discourse.julialang.org/u/davide445)\
**Post date:** [December 12, 2022, 12:51pm UTC](https://discourse.julialang.org/t/flux-with-amd-gpu-s/85218/19 "2022-12-12T12:51:43Z")

</div>

PyTorch support AMD GPU since more than one year

> **[PyTorch](https://pytorch.org/blog/pytorch-for-amd-rocm-platform-now-available-as-python-package/)**
>
> An open source machine learning framework that accelerates the path from research prototyping to production deployment.

I think this is not the differentiation factor Julia need to work on to be honest.  
Improve on this to gain broader adoption (also considering some top supercomputers does have AMD GPUs) for sure will be needed.

---

<div class="post-metadata">

**Author:** ![jpsamaroo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jpsamaroo/32/46804_2.png) [@jpsamaroo](https://discourse.julialang.org/u/jpsamaroo)\
**Post date:** [December 12, 2022, 6:15pm UTC](https://discourse.julialang.org/t/flux-with-amd-gpu-s/85218/20 "2022-12-12T18:15:25Z")

</div>

ML support for AMDGPU.jl has greatly improved thanks to efforts by @pxl-th and his colleagues. What we need now are more people with AMD GPUs to test things out, file issues when things are broken or badly performing (and PRs if possible), and more examples of how to use AMDGPU.jl for ML. Once we have enough people helping out, then it’ll be easier to keep up with the rest of the ML ecosystem and make Julia a top-notch competitor.

[Next page](https://discourse.julialang.org/t/flux-with-amd-gpu-s/85218.md?page=2)