Sm90 (H100) support for cuda.jl

smartalecH · March 29, 2023, 9:58pm

Does CUDA.jl not yet support Hopper gpus (sm90)?

I get the following error when trying to execute any kernels:

ERROR: LoadError: CUDA error: device kernel image is invalid (code 300, ERROR_INVALID_SOURCE)

Here’s my env:

julia> CUDA.versioninfo()
CUDA runtime 12.0, local installation
CUDA driver 12.1
NVIDIA driver 525.85.12, originally for CUDA 12.0

Libraries: 
- CUBLAS: 12.0.0
- CURAND: 10.3.1
- CUFFT: 11.0.0
- CUSOLVER: 11.4.2
- CUSPARSE: 12.0.0
- CUPTI: 18.0.0
- NVML: 12.0.0+525.85.12

Toolchain:
- Julia: 1.8.5
- LLVM: 13.0.1
- PTX ISA support: 3.2, 4.0, 4.1, 4.2, 4.3, 5.0, 6.0, 6.1, 6.3, 6.4, 6.5, 7.0, 7.1, 7.2
- Device capability support: sm_37, sm_50, sm_52, sm_53, sm_60, sm_61, sm_62, sm_70, sm_72, sm_75, sm_80, sm_86

Environment:
- JULIA_CUDA_MEMORY_POOL: none

8 devices:
  0: NVIDIA H100 (sm_90, 95.076 GiB / 95.577 GiB available)
  1: NVIDIA H100 (sm_90, 95.076 GiB / 95.577 GiB available)
  2: NVIDIA H100 (sm_90, 95.076 GiB / 95.577 GiB available)
  3: NVIDIA H100 (sm_90, 95.076 GiB / 95.577 GiB available)
  4: NVIDIA H100 (sm_90, 95.076 GiB / 95.577 GiB available)
  5: NVIDIA H100 (sm_90, 95.076 GiB / 95.577 GiB available)
  6: NVIDIA H100 (sm_90, 95.076 GiB / 95.577 GiB available)
  7: NVIDIA H100 (sm_90, 95.076 GiB / 95.577 GiB available)

I can see that sm90 is not listed in Device capability support… what needs to be done to support it?

carstenbauer · March 30, 2023, 5:02am

First thing I would do is try everything on Julia 1.9 (which comes with a newer LLVM) and with the latest CUDA.jl.

smartalecH · March 30, 2023, 5:33pm

Thanks @carstenbauer. Unfortunately, even with Julia 1.9 and using CUDA.jl from it’s main branch on Github, I still have the same issue (no support for sm_90).

smartalecH · March 30, 2023, 5:43pm

Hmm it looks like it requires LLVM v16 (the most recent julia 1.9 binaries ship with 14):

github.com

JuliaGPU/CUDA.jl/blob/aca936bc546ff07b851e0d3b803e96c7da46a585/src/compatibility.jl#L141-L143


      
          v"8.7" => v"16.0":highest,
          v"8.9" => v"16.0":highest,
          v"9.0" => v"16.0":highest,

I suppose I could build Julia using a newer version of LLVM (and apply the patches):

github.com

JuliaLang/julia/blob/ceafd6cad9a66e290aece75a62ba7733e03ba58d/doc/src/devdocs/build/build.md?plain=1#L245


      
          
          
The most complicated dependency is LLVM, for which we require additional patches from upstream (LLVM is not backward compatible).
          
          
For packaging Julia with LLVM, we recommend either:
           - bundling a Julia-only LLVM library inside the Julia package, or
           - adding the patches to the LLVM package of the distribution.
             * A complete list of patches is available in `deps/llvm.mk`, and the patches themselves are in `deps/patches/`.
             * The only Julia-specific patch is the lib renaming (`llvm-symver-jlprefix.patch`), which should _not_ be applied to a system LLVM.
             * The remaining patches are all upstream bug fixes, and have been contributed into upstream LLVM.
          
          
Using an unpatched or different version of LLVM will result in errors and/or poor performance. Though Julia can be built with newer LLVM versions, support for this should be regarded as experimental and not suitable for packaging.
          
          
### libuv
          
          
Julia uses a custom fork of libuv. It is a small dependency, and can be safely bundled in the same package as Julia, and will not conflict with the system library. Julia builds should _not_ try to use the system libuv.
          
          
### BLAS and LAPACK
          
          
As a high-performance numerical language, Julia should be linked to a multi-threaded BLAS and LAPACK, such as OpenBLAS or ATLAS, which will provide much better performance than the reference `libblas` implementations which may be default on some systems.
          
          
## Source distributions of releases

Before I do so, are there any other alternative options out there?