CUDAnative support for Float16

Specific Domains GPU
1

I want to perform Float16 computation on GPU. In the vadd.jl example, when I change the rand function type from Float32 to Float16, the following error happens while running:

ERROR: LoadError: InvalidIRError: compiling vadd(CuDeviceArray{Float16,2,CUDAnative.AS.Global}, CuDeviceArray{Float16,2,CUDAnative.AS.Global}, CuDeviceArray{Float16,2,CUDAnative.AS.Global}) resulted in invalid LLVM IR

Is there any other way to do Float16 computation with CUDAnative?

2

Any ideas?!

3

You should post in the GPU category, more people will spot your post that way.

You didn’t post the actual invalid IR, so it’s unclear what went wrong here. But IIRC Float16 is missing some bits in Base to work properly on GPUs, @vchuravy will know more.

4

I did not notice that GPU category exists. I change the category to GPU.

How can I get the actual invalid IR?

5

By writing GPU compatible code.

Please show a full MWE and all output. Please read: make it easier to help you

6

Float16 support in Julia is currently eagerly simulated in software by casting to and from Float32, and won’t be able use the hardware support that GPU have. There are many things necessary before we can fully use Float16 and the first step is https://github.com/JuliaLang/julia/pull/26381, but nobody has cared enough to invest the time to make it happen.

1 Like