Unable to use AMDGPU.jl on RX6600

bpsomu · March 12, 2025, 10:30pm

I am trying to use AMDGPU.jl but it installs with several errors and does not work.

Operating Systems: Fedora and Endeavor OS
Julia version: 1.11.4 installed via the curl command on the Julia website

julia> using AMDGPU
┌ Warning: HSA runtime is unavailable, compilation and runtime functionality will be disabled.
└ @ AMDGPU ~/.julia/packages/AMDGPU/BgSqf/src/AMDGPU.jl:168
┌ Warning: HIP library is unavailable, HIP integration will be disabled.
└ @ AMDGPU ~/.julia/packages/AMDGPU/BgSqf/src/AMDGPU.jl:195
┌ Warning: rocBLAS is unavailable, functionality will be disabled.
└ @ AMDGPU ~/.julia/packages/AMDGPU/BgSqf/src/AMDGPU.jl:206
┌ Warning: rocSPARSE is unavailable, functionality will be disabled.
└ @ AMDGPU ~/.julia/packages/AMDGPU/BgSqf/src/AMDGPU.jl:206
┌ Warning: rocSOLVER is unavailable, functionality will be disabled.
└ @ AMDGPU ~/.julia/packages/AMDGPU/BgSqf/src/AMDGPU.jl:206
┌ Warning: rocRAND is unavailable, functionality will be disabled.
└ @ AMDGPU ~/.julia/packages/AMDGPU/BgSqf/src/AMDGPU.jl:206
┌ Warning: rocFFT is unavailable, functionality will be disabled.
└ @ AMDGPU ~/.julia/packages/AMDGPU/BgSqf/src/AMDGPU.jl:206
┌ Warning: MIOpen is unavailable, functionality will be disabled.
└ @ AMDGPU ~/.julia/packages/AMDGPU/BgSqf/src/AMDGPU.jl:206

Then i tried allocating a ROCArray:

julia> a = zeros(10, 10)
10×10 Matrix{Float64}:
 0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0
 0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0
 0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0
 0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0
 0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0
 0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0
 0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0
 0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0
 0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0
 0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0  0.0

julia> b = ROCArray(a)
ERROR: could not load symbol "hipDeviceGet":
/home/bpsomu/.julia/juliaup/julia-1.11.4+0.x64.linux.gnu/bin/julia: undefined symbol: hipDeviceGet
Stacktrace:
  [1] macro expansion
    @ ~/.julia/packages/AMDGPU/BgSqf/src/hip/call.jl:38 [inlined]
  [2] macro expansion
    @ ~/.julia/packages/AMDGPU/BgSqf/src/utils.jl:122 [inlined]
  [3] hipDeviceGet(device::Base.RefValue{Ptr{Nothing}}, ordinal::Int64)
    @ AMDGPU.HIP ~/.julia/packages/AMDGPU/BgSqf/src/hip/libhip.jl:14
  [4] HIPDevice(device_id::Int64)
    @ AMDGPU.HIP ~/.julia/packages/AMDGPU/BgSqf/src/hip/device.jl:15
  [5] TaskLocalState
    @ ~/.julia/packages/AMDGPU/BgSqf/src/tls.jl:11 [inlined]
  [6] #25
    @ ~/.julia/packages/AMDGPU/BgSqf/src/tls.jl:27 [inlined]
  [7] get!(default::AMDGPU.var"#25#26"{Tuple{}}, d::IdDict{Any, Any}, key::Any)
    @ Base ./iddict.jl:171
  [8] task_local_state!()
    @ AMDGPU ~/.julia/packages/AMDGPU/BgSqf/src/tls.jl:26
  [9] device
    @ ~/.julia/packages/AMDGPU/BgSqf/src/tls.jl:45 [inlined]
 [10] ROCArray{Float64, 2, AMDGPU.Runtime.Mem.HIPBuffer}(::UndefInitializer, dims::Tuple{Int64, Int64})
    @ AMDGPU ~/.julia/packages/AMDGPU/BgSqf/src/array.jl:9
 [11] ROCArray
    @ ~/.julia/packages/AMDGPU/BgSqf/src/array.jl:111 [inlined]
 [12] ROCArray
    @ ~/.julia/packages/AMDGPU/BgSqf/src/array.jl:116 [inlined]
 [13] ROCArray(A::Matrix{Float64})
    @ AMDGPU ~/.julia/packages/AMDGPU/BgSqf/src/array.jl:119
 [14] top-level scope
    @ REPL[5]:1

Is it because my GPU RX6600 is unsupported by ROCm right now? If so, is there a way to still use hacks to get it to run?

ufechner7 · March 12, 2025, 10:42pm

I am using this command in my .bashrc file to make AMDGPU work with a Ryzen 7840U CPU which is not officially supported:

export HSA_OVERRIDE_GFX_VERSION=11.0.2

A similar command might work for you.

Try:

export HSA_OVERRIDE_GFX_VERSION=10.3.0

See: Support for RX6600XT (gfx1032,navi23)? · Issue #1698 · ROCm/ROCm · GitHub

bpsomu · March 12, 2025, 10:57pm

I used the following guide to install ROCm on my Linux distro:

gist.github.com

https://gist.github.com/augustin-laurent/d29f026cdb53a4dff50a400c129d3ea7

rocm_arch_guide.md


    Date of the guide : December 17, 2023

## Introduction

In this post, I will provide the solution that worked on my system on how to install Radeon Open Compute (ROCm) on Arch (linux-6.6.7.arch1-1) for RX 6900 XT (Should work on other 6000 series). 
ROCm is an open-source software platform that allows GPU-accelerated computation. 
This tool is a prerequist to use GPU Acceleration on TensorFlow or PyTorch.
In this guide I will use Paru as my AUR package helper, feel free to use any other (https://wiki.archlinux.org/title/AUR_helpers).
I will assume you have a working operating system and know what you do with it (Otherwise Arch will be painfull for you).

This file has been truncated. show original

Which also included adding

export HSA_OVERRIDE_GFX_VERSION=10.3.0

now, AMDGPU.jl works on Arch. Thanks. It still does not work on Fedora though.

bpsomu · March 12, 2025, 11:39pm

While the fix you suggested worked on my Arch linux installation after I installed ROCm from the ArchUserRepository (AUR). It did not work on my Fedora install even after I installed ROCm through the dnf package manager. Still looking for a fix there.

lxmota · March 18, 2025, 1:53am

I have the same issue on Fedora 41 with a RX 7600 GPU. Setting the variable HSA_OVERRIDE_GFX_VERSION to 11.0.2 or 10.3.0 seems to have no effect:

$ env HSA_OVERRIDE_GFX_VERSION=10.3.0 julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.11.4 (2025-03-10)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> using Pkg

julia> Pkg.build("AMDGPU")

julia> using AMDGPU
┌ Warning: HSA runtime is unavailable, compilation and runtime functionality will be disabled.
└ @ AMDGPU ~/.julia/packages/AMDGPU/VTHhJ/src/AMDGPU.jl:168
┌ Warning: HIP library is unavailable, HIP integration will be disabled.
└ @ AMDGPU ~/.julia/packages/AMDGPU/VTHhJ/src/AMDGPU.jl:195
┌ Warning: rocBLAS is unavailable, functionality will be disabled.
└ @ AMDGPU ~/.julia/packages/AMDGPU/VTHhJ/src/AMDGPU.jl:206
┌ Warning: rocSPARSE is unavailable, functionality will be disabled.
└ @ AMDGPU ~/.julia/packages/AMDGPU/VTHhJ/src/AMDGPU.jl:206
┌ Warning: rocSOLVER is unavailable, functionality will be disabled.
└ @ AMDGPU ~/.julia/packages/AMDGPU/VTHhJ/src/AMDGPU.jl:206
┌ Warning: rocRAND is unavailable, functionality will be disabled.
└ @ AMDGPU ~/.julia/packages/AMDGPU/VTHhJ/src/AMDGPU.jl:206
┌ Warning: rocFFT is unavailable, functionality will be disabled.
└ @ AMDGPU ~/.julia/packages/AMDGPU/VTHhJ/src/AMDGPU.jl:206
┌ Warning: MIOpen is unavailable, functionality will be disabled.
└ @ AMDGPU ~/.julia/packages/AMDGPU/VTHhJ/src/AMDGPU.jl:206

ROCm itself seems to work according to C++ tests.

pxl-th · March 18, 2025, 11:58am

Where is the ROCm located on your system? AMDGPU.jl fails to find it

bpsomu · March 18, 2025, 12:07pm

I removed my Fedora install and installed Debian which is apparently officially supported by AMD. I followed the official installation instructions and it works for me now.

lxmota · March 18, 2025, 8:30pm

Headers are installed on /usr/include, executables on /usr/bin and libraries on /usr/lib64.

lxmota · March 19, 2025, 1:47am

Additional info that might be useful:

julia> using Libdl

julia> Libdl.dlopen("librocblas")
     17336:	find library=librocblas [0]; searching
     17336:	 search path=/home/mota/.julia/juliaup/julia-1.11.4+0.x64.linux.gnu/bin/../lib/julia:/home/mota/.julia/juliaup/julia-1.11.4+0.x64.linux.gnu/bin/../lib/julia/..		(RUNPATH from file /home/mota/.julia/juliaup/julia-1.11.4+0.x64.linux.gnu/bin/julia)
     17336:	  trying file=/home/mota/.julia/juliaup/julia-1.11.4+0.x64.linux.gnu/bin/../lib/julia/librocblas
     17336:	  trying file=/home/mota/.julia/juliaup/julia-1.11.4+0.x64.linux.gnu/bin/../lib/julia/../librocblas
     17336:	 search cache=/etc/ld.so.cache
     17336:	 search path=/lib64:/usr/lib64		(system search path)
     17336:	  trying file=/lib64/librocblas
     17336:	  trying file=/usr/lib64/librocblas
     17336:	
     17336:	find library=librocblas.so [0]; searching
     17336:	 search path=/home/mota/.julia/juliaup/julia-1.11.4+0.x64.linux.gnu/bin/../lib/julia:/home/mota/.julia/juliaup/julia-1.11.4+0.x64.linux.gnu/bin/../lib/julia/..		(RUNPATH from file /home/mota/.julia/juliaup/julia-1.11.4+0.x64.linux.gnu/bin/julia)
     17336:	  trying file=/home/mota/.julia/juliaup/julia-1.11.4+0.x64.linux.gnu/bin/../lib/julia/librocblas.so
     17336:	  trying file=/home/mota/.julia/juliaup/julia-1.11.4+0.x64.linux.gnu/bin/../lib/julia/../librocblas.so
     17336:	 search cache=/etc/ld.so.cache
     17336:	  trying file=/usr/lib64/rocm/gfx1100/lib/librocblas.so
     17336:	
     17336:	find library=libamdhip64.so.6 [0]; searching
     17336:	 search cache=/etc/ld.so.cache
     17336:	  trying file=/lib64/libamdhip64.so.6
     17336:	
     17336:	find library=libamd_comgr.so.2 [0]; searching
     17336:	 search cache=/etc/ld.so.cache
     17336:	  trying file=/lib64/libamd_comgr.so.2
     17336:	
     17336:	find library=libhsa-runtime64.so.1 [0]; searching
     17336:	 search cache=/etc/ld.so.cache
     17336:	  trying file=/lib64/libhsa-runtime64.so.1
     17336:	
     17336:	find library=libnuma.so.1 [0]; searching
     17336:	 search cache=/etc/ld.so.cache
     17336:	  trying file=/lib64/libnuma.so.1
     17336:	
     17336:	find library=liblldELF.so.18.1 [0]; searching
     17336:	 search cache=/etc/ld.so.cache
     17336:	  trying file=/usr/lib64/llvm18/lib/liblldELF.so.18.1
     17336:	
     17336:	find library=liblldCommon.so.18.1 [0]; searching
     17336:	 search cache=/etc/ld.so.cache
     17336:	  trying file=/usr/lib64/llvm18/lib/liblldCommon.so.18.1
     17336:	
     17336:	find library=libclang-cpp.so.18.1 [0]; searching
     17336:	 search cache=/etc/ld.so.cache
     17336:	  trying file=/usr/lib64/llvm18/lib/libclang-cpp.so.18.1
     17336:	
     17336:	find library=libLLVM.so.18.1 [0]; searching
     17336:	 search cache=/etc/ld.so.cache
     17336:	  trying file=/usr/lib64/llvm18/lib/libLLVM.so.18.1
     17336:	
     17336:	find library=libelf.so.1 [0]; searching
     17336:	 search cache=/etc/ld.so.cache
     17336:	  trying file=/lib64/libelf.so.1
     17336:	
     17336:	find library=libhsakmt.so.1 [0]; searching
     17336:	 search cache=/etc/ld.so.cache
     17336:	  trying file=/lib64/libhsakmt.so.1
     17336:	
     17336:	find library=libdrm.so.2 [0]; searching
     17336:	 search cache=/etc/ld.so.cache
     17336:	  trying file=/lib64/libdrm.so.2
     17336:	
     17336:	find library=libffi.so.8 [0]; searching
     17336:	 search cache=/etc/ld.so.cache
     17336:	  trying file=/lib64/libffi.so.8
     17336:	
     17336:	find library=libedit.so.0 [0]; searching
     17336:	 search cache=/etc/ld.so.cache
     17336:	  trying file=/lib64/libedit.so.0
     17336:	
     17336:	find library=libtinfo.so.6 [0]; searching
     17336:	 search cache=/etc/ld.so.cache
     17336:	  trying file=/lib64/libtinfo.so.6
     17336:	
     17336:	find library=libzstd.so.1 [0]; searching
     17336:	 search cache=/etc/ld.so.cache
     17336:	  trying file=/lib64/libzstd.so.1
     17336:	
     17336:	find library=libdrm_amdgpu.so.1 [0]; searching
     17336:	 search cache=/etc/ld.so.cache
     17336:	  trying file=/lib64/libdrm_amdgpu.so.1
     17336:	
     17336:	
     17336:	calling init: /lib64/libdrm.so.2
     17336:	
     17336:	
     17336:	calling init: /lib64/libdrm_amdgpu.so.1
     17336:	
     17336:	
     17336:	calling init: /lib64/libzstd.so.1
     17336:	
     17336:	
     17336:	calling init: /lib64/libtinfo.so.6
     17336:	
     17336:	
     17336:	calling init: /lib64/libedit.so.0
     17336:	
     17336:	
     17336:	calling init: /lib64/libffi.so.8
     17336:	
     17336:	
     17336:	calling init: /lib64/libnuma.so.1
     17336:	
     17336:	
     17336:	calling init: /lib64/libhsakmt.so.1
     17336:	
     17336:	
     17336:	calling init: /lib64/libelf.so.1
     17336:	
     17336:	
     17336:	calling init: /usr/lib64/llvm18/lib/libLLVM.so.18.1
     17336:	
     17336:	
     17336:	calling init: /usr/lib64/llvm18/lib/libclang-cpp.so.18.1
     17336:	
     17336:	
     17336:	calling init: /usr/lib64/llvm18/lib/liblldCommon.so.18.1
     17336:	
     17336:	
     17336:	calling init: /usr/lib64/llvm18/lib/liblldELF.so.18.1
     17336:	
     17336:	
     17336:	calling init: /lib64/libhsa-runtime64.so.1
     17336:	
     17336:	
     17336:	calling init: /lib64/libamd_comgr.so.2
     17336:	
     17336:	
     17336:	calling init: /lib64/libamdhip64.so.6
     17336:	
     17336:	
     17336:	calling init: /usr/lib64/rocm/gfx1100/lib/librocblas.so
     17336:	
     17336:	/home/mota/.julia/juliaup/julia-1.11.4+0.x64.linux.gnu/bin/../lib/julia/libjulia-internal.so.1.11: error: symbol lookup error: undefined symbol: ijl_gc_enable_finalizers_internal (fatal)
     17336:	/home/mota/.julia/juliaup/julia-1.11.4+0.x64.linux.gnu/bin/../lib/julia/libjulia-internal.so.1.11: error: symbol lookup error: undefined symbol: ijl_gc_enable_finalizers_internal (fatal)
     17336:	/home/mota/.julia/juliaup/julia-1.11.4+0.x64.linux.gnu/bin/../lib/julia/libjulia-internal.so.1.11: error: symbol lookup error: undefined symbol: ijl_gc_have_pending_finalizers (fatal)
     17336:	/home/mota/.julia/juliaup/julia-1.11.4+0.x64.linux.gnu/bin/../lib/julia/libjulia-internal.so.1.11: error: symbol lookup error: undefined symbol: ijl_gc_have_pending_finalizers (fatal)
     17336:	/home/mota/.julia/juliaup/julia-1.11.4+0.x64.linux.gnu/bin/../lib/julia/libjulia-internal.so.1.11: error: symbol lookup error: undefined symbol: ijl_gc_run_pending_finalizers (fatal)
     17336:	/home/mota/.julia/juliaup/julia-1.11.4+0.x64.linux.gnu/bin/../lib/julia/libjulia-internal.so.1.11: error: symbol lookup error: undefined symbol: ijl_gc_run_pending_finalizers (fatal)
Ptr{Nothing} @0x00000000350bb720

It appears that Julia finds the correct librocblas.so located in /usr/lib64/rocm/gfx1100/lib/librocblas.so, but there are some undefined symbols related to the garbage collector?

pxl-th · March 19, 2025, 6:13pm

Can you try setting env variable ROCM_PATH=/usr/lib64/rocm/gfx1100 and then launching Julia?

lxmota · March 19, 2025, 8:31pm

Now I get this:

$ ROCM_PATH=/usr/lib64/rocm/gfx1100 julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.11.4 (2025-03-10)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> using AMDGPU
┌ Warning: HSA runtime is unavailable, compilation and runtime functionality will be disabled.
└ @ AMDGPU ~/.julia/packages/AMDGPU/VTHhJ/src/AMDGPU.jl:168
┌ Warning: HIP library is unavailable, HIP integration will be disabled.
└ @ AMDGPU ~/.julia/packages/AMDGPU/VTHhJ/src/AMDGPU.jl:195

So two warnings left. Could be because the HSA and HIP libraries are located in /usr/lib64. Next I tried this:

$ ROCM_PATH=/usr/lib64/rocm/gfx1100 HSA_PATH=/usr/lib64 HIP_PATH=/usr/lib64 julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.11.4 (2025-03-10)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> using AMDGPU
┌ Warning: HSA runtime is unavailable, compilation and runtime functionality will be disabled.
└ @ AMDGPU ~/.julia/packages/AMDGPU/VTHhJ/src/AMDGPU.jl:168
┌ Warning: HIP library is unavailable, HIP integration will be disabled.
└ @ AMDGPU ~/.julia/packages/AMDGPU/VTHhJ/src/AMDGPU.jl:195

julia>

But I still get the warnings about HSA and HIP.

pxl-th · March 19, 2025, 8:35pm

Can you find where libhsa-runtime64.so and libamdhip64.so are located? You can then symlink them to /usr/lib64/rocm/gfx1100 and retry

lxmota · March 19, 2025, 10:31pm

I created the symlinks:

root@sirius:/usr/lib64/rocm/gfx1100# locate libhsa-runtime64.so
/usr/lib64/libhsa-runtime64.so
/usr/lib64/libhsa-runtime64.so.1
/usr/lib64/libhsa-runtime64.so.1.14.0
root@sirius:/usr/lib64/rocm/gfx1100# ln -s /usr/lib64/libhsa-runtime64.so .
root@sirius:/usr/lib64/rocm/gfx1100# ln -s /usr/lib64/libhsa-runtime64.so.1 .
root@sirius:/usr/lib64/rocm/gfx1100# ln -s /usr/lib64/libhsa-runtime64.so.1.14.0 .
root@sirius:/usr/lib64/rocm/gfx1100# locate libamdhip64.so
/usr/lib64/libamdhip64.so
/usr/lib64/libamdhip64.so.6
/usr/lib64/libamdhip64.so.6.2.41134
root@sirius:/usr/lib64/rocm/gfx1100# ln -s /usr/lib64/libamdhip64.so .
root@sirius:/usr/lib64/rocm/gfx1100# ln -s /usr/lib64/libamdhip64.so.6 .
root@sirius:/usr/lib64/rocm/gfx1100# ln -s /usr/lib64/libamdhip64.so.6.2.41134 .

and I get the exact same two warnings, which is weird. I tried reinstalling AMDGPU.jl from scratch and I still got those two last warnings about HSA and HIP:

julia> using AMDGPU
┌ Warning: HSA runtime is unavailable, compilation and runtime functionality will be disabled.
└ @ AMDGPU ~/.julia/packages/AMDGPU/VTHhJ/src/AMDGPU.jl:168
┌ Warning: HIP library is unavailable, HIP integration will be disabled.
└ @ AMDGPU ~/.julia/packages/AMDGPU/VTHhJ/src/AMDGPU.jl:195

lxmota · March 19, 2025, 10:40pm

My mistake! The symlinks should go in:

/usr/lib64/rocm/gfx1100

Now I get no warnings:

julia> using AMDGPU

julia> AMDGPU.functional()
true

Topic		Replies	Views
Using AMDGPU.jl with an AMD APU GPU	4	916	June 2, 2022
Some AMDGPU tests failing on Radeon VII under Windows GPU amdgpu	10	631	December 27, 2023
Failed to precompile OpenCL in julia V1.9, for GPU programming, AMD Radeon graphics New to Julia question , package	13	559	December 1, 2023
Unable to test or use HIP libraries on Windows 10 GPU amdgpu	4	1136	February 2, 2024
Failed while initializing AMDGPU.jl with LLVM 17 and ROCm 6.1 on Fedora 40 New to Julia amdgpu	4	211	September 18, 2024

Unable to use AMDGPU.jl on RX6600

Related topics