What is the current status of GPU support in Julia?

General Usage
1

According to the official website, only NVIDIA GPUs using CUDA on Linux is offered tier 1 support. AMD GPUs using ROCm is only offered tier 3 support. However, the AMDGPU.jl repository does not have any warnings in the documentation, which indicates it should be usable? Then, what is the current status of GPU support for different GPUs in Julia?

2

unclear whatā€™s the logic here. And you should really just try to see your workload works.

I think itā€™s safe to say CUDA.jl has the best support still, but the difference may not matter to your specific use case already.

1 Like
3

I was rather shocked to see that statement on the Julialang website.
Looking at juliagpu.org
AMDGPU.jl offers similar capabilities for AMD GPUs running on the ROCm stack. While somewhat less developed than CUDA.jl, it provides solid integration with key vendor libraries such as rocBLAS, rocFFT, rocSOLVER, and rocSPARSE. The package can be slightly more challenging to install and configure due to how ROCm is distributed, but it has been successfully used in various applications and libraries.

Maybe you can say a bit more about your application. Are you at the stage of choosing a platform?

I suppose I best decloak as a principal engineer at AMD

4 Likes
4

ps the link on julialang.org should be pointing to AMD ROCm ā‹… JuliaGPU

5

If you use AMDGPU.jl, Iā€™m shocked that youā€™re shocked.

Tier 3 is pretty reflective of my personal experience with AMDGPU.jl. I almost always need to access my desktop machine with an Nvidia GPU when I want to test something since AMDGPU.jl is broken more often than not on recent julia releases if one is using a reasonably up-to-date kernel and other stack.

E.g. for an example of what I mean, I just tried it out and calling stuff like AMDGPU.rand(10) or AMDGPU.versioninfo() both crash my julia session with a segfault. There appears to be someone who reported this three weeks ago here and has not had any support, help, or suggestions (let alone bugfixes).

Iā€™m sure it does work sometimes and on some configurations of machines (Iā€™ve even succesfully gotten it working on my own machine a few times). But the description of Tier 3 is just

Tier 3: Julia may or may not build. If it does, it is unlikely to pass tests. Binaries may be available in some cases. When they are, they should be considered experimental. Ongoing support is dependent on community efforts.

So ā€œit sometimes working for some people, but with no guarantees itā€™ll work for you or pass testsā€ is definitely an accurate description of my experience.

4 Likes
6

I should have said ā€˜disappointedā€™

7

AMDGPU.jl only tests against officially supported linux distributions (Ubuntu/Debian to be specific) and on those we have a very solid CI that passes all the tests for a very long time now.
That said, we only have 2 CI machines with Navi 3 GPUs, so other distributions and GPUs may vary.
Other linux distributions have custom build recipes that do not always match official ones and that may conflct with Julia and we donā€™t have the resources to test against all possible builds.

E.g. for an example of what I mean, I just tried it out and calling stuff like AMDGPU.rand(10) or AMDGPU.versioninfo() both crash my julia session with a segfault. There appears to be someone who reported this three weeks ago here and has not had any support, help, or suggestions (let alone bugfixes).

The original author has since resolved his issues with a message on slack saying that reinstalling libraries fixed it for him:

it is working after swtihing from rhel packages to fedora ones

1 Like
8

These tiers are generally descriptive of the current state of affairs rather than any indication of how weā€™d like things to beā€”obviously weā€™d like support to be as good as possible everywhere. All thatā€™s really needed to increase the tier of some platform is for someone to fix builds, fix tests, and help with making sure we have CI resources to keep it that way. Often this last item is the real blocker. We are, for what itā€™s worth, happy to have physical hardware running in a data center (or closet somewhere) for this purpose and a small amount of physical dedicated hardware goes a long way.

5 Likes
9

Sure, and itā€™s totally fine for AMDGPU.jl to (severely) limit the linux distros it targets, and to have limited CI configurations and all that (even if I wish it wasnā€™t so limited).

Iā€™m just saying that this sort of thing is indicative of why this wouldnā€™t be labelled as ā€œTier 1 supportā€ on juliaā€™s own website, which as far as I understand is a very high bar, and leaves a lot of ā€˜liabilityā€™ on the julia devs themselves.

(IMO the tier support labelling of these GPU packages is kinda a weird thing for the julia install page to do at all, but thatā€™s a different conversation.)

2 Likes
10

Iā€™m now preparing for a new project, which is supposed to be run on both CPUs and GPUs, and it will be a great bonus if it can support different GPUs.
My previous project, which does not need to support GPUs, is developed with Julia, and generally speaking Iā€™m quite pleased with the develop process. Thus, I tend to continue to use Julia for my new project. Iā€™ve also considered that even if Julia itself does not perform well on some GPUs, I can write the problematic part in C and call the library in Julia.
Thus, it is really disappointing that even Julia itself is supported so poorly in AMD GPUs, and a simple call like rand or even versioninfo will crash it. It is good news that at least Ubuntu and Debian is supported well, but the support for Centos is very important in my use case.
That being said, Iā€™m now seriously considering to develop the new project in C or Fortran. That project is large and may take years to develop, but Iā€™m not sure whether the support will be better before it is completed.

Iā€™ve checked the posts again carefully, and I must say that I have limited knowledge to GPUs. It seems that AMD itself has limited support for distributions? Besides, what breaks Julia is in fact methods in AMDGPU.jl, so maybe it is still possible that I write CPU-related parts in Julia, GPU-related parts in C, and call C libraries from Julia. Will it work?

11

but the support for Centos is very important in my use case

CentOS itself has limited support by AMD, so you should just test first to see if it works.
Latest ROCm to support it is 6.1 (6.4 being the latest version).

Julia itself is supported so poorly in AMD GPUs, and a simple call like rand or even versioninfo will crash it

If ROCm libraries were built incorrectly they will crash on the first API call.

I have built fairly complex Julia projects that taget different GPUs in a backend agnostic way and have great performance on both AMD & Nvidia GPUs:

Same goes for other projects that also target multiple GPUs:

5 Likes
12

Yes

You definitely can write CPU part in Julia, even target specific GPU backends without being affected by issues from other GPU backends. And even if you do encounter issues, opening a bug report may help

1 Like
13

@jsjie do you know about KernelAbstractions.jl? It will let you just write one GPU kernel and have it work on Nvidia, Apple, AMD, or Intel GPUs depending on what GPU the user has (or run multithreaded on the CPU).

To be clear, itā€™s not especially poorly supported. It will for the most part either ā€œjust workā€ or ā€œjust failā€ depending on how you system and how it was installed.

Hopefully AMD will eventually realize the value of supporting more distros, but since CentOS was completely discontinued, I wouldnā€™t expect that particular distro to ever get active support work. Maybe you can get it to work by making an Ubuntu docker container or chroot though. No idea.

I mean this might work, but I donā€™t really see the point. If AMDGPU.jl doesnā€™t work on your machine, the C libraries are also pretty unlikely to work on your machine. Itā€™s really just a systemic issue with AMD GPU compute, not the specific instantiation in Julia.

1 Like
14

In Computex AMD promised support for Fedora, OpenSUSE, Ubuntu and EPEL repos.

The Fedora support is the one in Fedora repos. OpenSUSE is probably this, but it is still experimental. The Ubuntu support most likely means that in the future ROCm comes to the default Ubuntu repos.

I understood that the main intend is that you can install ROCm in the future with just sudo apt/dnf/zypper install rocm.

1 Like
16

For installing ROCm I use the amdgpu-installer script
It uses the package manager.

It also sets up /etc/alternatives - which I am no great fan of. But you dont have to use that mechanism.

17

The Runfile installer may be useful for Julia package management
The ROCm Runfile Installer includes these features:

  • An optional easy-to-use user interface for configuring the installation
  • An optional command line interface for the installation
  • Offline ROCm and AMDGPU driver installation (requires the prior installation of dependencies)
  • Packageless ROCm and AMDGPU driver install without native package management
  • A single self-contained installer for all ROCm and AMDGPU driver software
  • Configurable installation location for the ROCm install