Why Julia is much slower than MATLAB on GPU computing?

Specific Domains GPU
,
21

Interesting, I wonder why it’s so slow on your machine then, especially compared to my much slower GPU.

Can you show us your CUDA.versioninfo()?

22

Looking at your script, you only have the CUDA.@time line once. When you say run it several times, are you 1) running that line repeatedly in the same process, 2) includeing the whole script repeatedly in the same process, 3) running different processes, or 4) something else? Just for clarification.

2 Likes
23

I run as follows and skip the 1st iteration result.

for _ in 1:10
     @show CUDA.@time CUDA.@sync V, idx, iter = main(2^13)
end
1 Like
24 
julia> CUDA.versioninfo()
CUDA runtime 12.1, artifact installation
CUDA driver 12.2
Unknown NVIDIA driver

CUDA libraries:
- CUBLAS: 12.1.3
- CURAND: 10.3.2
- CUFFT: 11.0.2
- CUSOLVER: 11.4.5
- CUSPARSE: 12.1.0
- CUPTI: 18.0.0
- NVML: missing

Julia packages:
- CUDA: 4.4.1
- CUDA_Driver_jll: 0.5.0+1
- CUDA_Runtime_jll: 0.6.0+0

Toolchain:
- Julia: 1.9.3
- LLVM: 14.0.6
- PTX ISA support: 3.2, 4.0, 4.1, 4.2, 4.3, 5.0, 6.0, 6.1, 6.3, 6.4, 
6.5, 7.0, 7.1, 7.2, 7.3, 7.4, 7.5
- Device capability support: sm_37, sm_50, sm_52, sm_53, sm_60, sm_61, sm_62, sm_70, sm_72, sm_75, sm_80, sm_86

1 device:
  0: NVIDIA GeForce RTX 3090 (sm_86, 20.118 GiB / 23.999 GiB available)
25

That loop would definitely work for the base @show @time. Could you amend your original post’s script with the loop and also provide the printout? Actually, you might not want to paste the arrays over and over, so for brevity you could omit the @show, just showing the timings.

1 Like
26

I noticed that you use Linux, while I use Windows. I am not sure whether the operating system may influence the speed.

In my previous experiments, I ran the original example on a workstation using one NVIDIA RTX 3090 with a Linux system and also on my local Windows system computer. I found that for GPU computing, the runtime of MATLAB and Julia is similar on the Linux machine, whereas Julia is far slower than MATLAB on the Windows computer.

Here are some influential factors affecting speed that need to be checked:
(workstation vs. local computer)

  1. Operating system: Linux vs. Windows
  2. CPU memory size: 256 GB vs. 32 GB

However, due to some limitations, I cannot install MATLAB 2023a on the Linux workstation, so I returned to my local machine. I am still trying, but currently, I am unable to install Linux on my computer.

1 Like
27

CUDA.jl has actual tools to figure out what part of your application is slow, as I already mentioned in Why Julia is much slower than MATLAB on GPU computing? - #4 by maleadt. You’re also using an outdated version of CUDA.jl, missing out on bugfixes and performance improvements that have been made since, so you should re-test using CUDA.jl v5.1.1.

4 Likes
28

Definitely try updating to a newer version of CUDA.

Edit: I missed that maleadt posted again before me. I strongly recommend you just follow his advice, I never understood why you just ignored his original post. It was full of useful advice for how to actually track this down.

29

Thanks. I try to update the CUDA.jl but still fail at present.

30

I am trying to update the CUDA version. I guess maybe I did not install the whole CUDA environment successfully in the past, for example, CUDA.versioninfo() does not show NVML .

31

That’s not important; NVML is just used to print some additional information. You do not have to update CUDA, only CUDA.jl.

2 Likes