Peculiar GPU behavior: zero performance cost for math functions?

bremez
Fri, 03 Nov 2023 16:22:07 +0000
GPU

I have a use case where the performance bottleneck is in computing trigonometric functions. I thought that maybe it would benefit from some more parallelism, and realized that having access to a university computing cluster is a good time to learn GPU computing for the first time .

I found that surprisingly, not only do I benefit from the parallelism advantage, but also that seemingly, on the GPU the cost of computing trigonometric functions turns out to be negligible compared to trivial arithmetic. See the MWE example below where cos takes approximately the same time as x -> x^ to map over the GPU, but ~8 times as long on the CPU.

Could anyone shed any light on why that happens, and if maybe I can “backport” this behavior to my present CPU-implemented use case? My one guess is that maybe on the GPU the calculation is memory-bandwidth-limited in both calculations, but the memory bandwidth for this device is 900 GB/s and (N*sizeof(eltype(x))) / (900 * 2^30) predicts approximately 70 microseconds, and so it doesn’t seem to be the predominant bottleneck.

using CUDA, BenchmarkTools, Test

println(CUDA.name(device()))
# 	Tesla V100-SXM2-32GB

N = 2^24;
x_d = CUDA.rand(N) ;
x = collect(x_d);

CUDA.@sync map!(cos, x_d, x_d); map!(cos, x, x);
println(isapprox(x, collect(x_d)))
# 	true

@btime  CUDA.@sync map!(x -> x^2, $(x_d), $(x_d));
#	196.627 us (65 allocations: 2.95 KiB)
@btime  map!(x -> x^2, $(x), $(x));
#	11.456 ms (0 allocations: 0 bytes)

@btime  CUDA.@sync map!(cos, $(x_d), $(x_d));
#	203.606 us (65 allocations: 2.95 KiB) 	---> basically same as arithmatic
@btime  map!(cos, $(x), $(x));
# 	87.118 ms (0 allocations: 0 bytes)    	---> considerably slower than arithmetic

6 posts - 4 participants