# GPU Julia vs GPU Matlab

**URL:** <https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663>\
**Category:** New to Julia\
**Tags:** gpu\
**Created:** [November 15, 2024, 5:14am UTC](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663 "2024-11-15T05:14:56Z")\
**Posts on this page:** 20\
**Page:** 2

<div class="post-metadata">

**Author:** ![Alex90](https://avatars.discourse-cdn.com/v4/letter/a/6de8d8/32.png) [@Alex90](https://discourse.julialang.org/u/Alex90)\
**Post date:** [November 15, 2024, 10:22am UTC](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663/21 "2024-11-15T10:22:53Z")

</div>

I did this in my main program. Indeed, it improved performance. But still, the Matlab GPU performance was not reached

---

<div class="post-metadata">

**Author:** ![roflmaostc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/roflmaostc/32/30123_2.png) [@roflmaostc](https://discourse.julialang.org/u/roflmaostc)\
**Post date:** [November 15, 2024, 10:24am UTC](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663/22 "2024-11-15T10:24:04Z")

</div>

> [@Alex90](#):
>
> Matlab

can you please share your recent Matlab and Julia code. I want to try on my machine.

---

<div class="post-metadata">

**Author:** ![roflmaostc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/roflmaostc/32/30123_2.png) [@roflmaostc](https://discourse.julialang.org/u/roflmaostc)\
**Post date:** [November 15, 2024, 10:29am UTC](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663/23 "2024-11-15T10:29:03Z")

</div>

> [@Alex90](#):
>
> Matlab

I think matrix multiplication is excluded, in my tests the results were identical but yeah, some spurious small allocations. But it’s not a full copy just 160 Bytes.

```julia
function lel()
    A = rand(51,51,51);
    B = rand(A);
    C = zeros(51,51,51);
    D = similar(C);
    E = similar(C);
    F = similar(C);
    function math1!(C, A, B)
        @. C = A^2 + B^2 + A * B 
        @. C += A / B - A * B - A / B + A * B + A / B - A * B - A / B 
        return C
    end 

    function math2!(D, A, B)
        @. D = A^2 + B^2 + A * B + A / B - A * B - A / B + A * B + A / B - A * B - A / B 
        return D
    end 

    @time math1!(C, A, B)
    @time math2!(D, A, B)

    @show C ≈ D 
    return nothing
end

julia> lel()
  0.119883 seconds
  0.090662 seconds (8 allocations: 160 bytes)
C ≈ D = true

```

---

<div class="post-metadata">

**Author:** ![Alex90](https://avatars.discourse-cdn.com/v4/letter/a/6de8d8/32.png) [@Alex90](https://discourse.julialang.org/u/Alex90)\
**Post date:** [November 15, 2024, 10:37am UTC](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663/24 "2024-11-15T10:37:57Z")

</div>

Dear roflmaostc,  
Sure. The Julia code revised by danielwe is as follows:

```julia
A = CUDA.rand(151,151,151);
B = CUDA.rand(151,151,151);
C = CUDA.zeros(151,151,151);
D = similar(C);
E = similar(C);
F = similar(C);
function math1!(C, A, B)
    @. C = A^2 + B^2 + A * B + A / B - A * B - A / B + A * B + A / B - A * B - A / B
    return C
end
function math2!(D, C)
    @. D = C^2 + C^2 + C * C + C / C - C * C - C / C + C * C + C / C - C * C - C / C
    return D
end
function math3!(E, D)
    @. E = D^2 + D^2 + D * D + D / D - D * D - D / D + D * D + D / D - D * D - D / D
    return E
end
@btime for iter = 1:1000
    math1!($C, $A, $B)
    math2!($D, $C)
    math3!($E, $D)
end

```

The Matlab code is as follows:

```julia
A = (single(rand(151,151,151,'gpuArray')));
B = (single(rand(151,151,151,'gpuArray')));
C = (single(zeros(151,151,151,'gpuArray')));
D = (single(zeros(151,151,151,'gpuArray')));
E = (single(zeros(151,151,151,'gpuArray')));
tic
for iter = 1:1000
    % disp(iter)
    C = A.^2 + B.^2 + A .* B + A ./ B - A .* B - A ./ B + A .* B + A ./ B - A .* B - A ./ B;
    D = C.^2 + C.^2 + C .* C + C ./ C - C .* C - C ./ C + C .* C + C ./ C - C .* C - C ./ C;
     E= D.^2 + D.^2 + D .* D + D ./ D - D .* D - D ./ D + D .* D + D ./ D - D .* D - D ./ D;
end
toc

```

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [November 15, 2024, 10:48am UTC](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663/25 "2024-11-15T10:48:48Z")

</div>

That code is 64bits in Julia still.

---

<div class="post-metadata">

**Author:** ![roflmaostc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/roflmaostc/32/30123_2.png) [@roflmaostc](https://discourse.julialang.org/u/roflmaostc)\
**Post date:** [November 15, 2024, 11:38am UTC](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663/26 "2024-11-15T11:38:21Z")

</div>

> [@Alex90](#):
>
> `A = CUDA.rand(151,151,151);`

This returns a `Float32` eltype. So it should be good.

---

<div class="post-metadata">

**Author:** ![roflmaostc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/roflmaostc/32/30123_2.png) [@roflmaostc](https://discourse.julialang.org/u/roflmaostc)\
**Post date:** [November 15, 2024, 12:32pm UTC](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663/27 "2024-11-15T12:32:38Z")

</div>

Thanks. On my RTX 3060 and Matlab R2023a and my Julia 1.10.6 and Julia - CUDA: 5.5.2 and CUDA runtime 12.CUDA runtime 12.6 I obverse ~0.58s for Matlab and around 1.5s for Julia.

I also removed all divisions by zero (random matrices with entries \> 0) but this did not make any difference. I also check if Matlab might eliminate the trivial `A .* B .- A .* B` operations, but seems like it doesn’t.

One minor mistake is to [not wait for the device in Matlab](https://ch.mathworks.com/matlabcentral/answers/538991-confusing-results-when-measuring-time-of-function-on-cpu-gpu-with-tic-toc-gpu-timeit#answer_444111).

With that I get Matlab=0.85s and Julia=1.4s

```matlab
A = (single(rand(151,151,151,'gpuArray'))) + 1;
B = (single(rand(151,151,151,'gpuArray'))) + 1;
C = (single(zeros(151,151,151,'gpuArray'))) + 1;
D = (single(zeros(151,151,151,'gpuArray')));
E = (single(zeros(151,151,151,'gpuArray')));

dev = gpuDevice()
timer = tic()

for iter = 1:1000
    % disp(iter)
    C = A.^2 + B.^2 + A .* B + A ./ B - A .* B - A ./ B + A .* B + A ./ B - A .* B - A ./ B;
    D = C.^2 + C.^2 + C .* C + C ./ C - C .* C - C ./ C + C .* C + C ./ C - C .* C - C ./ C;
    E = D.^2 + D.^2 + D .* D + D ./ D - D .* D - D ./ D + D .* D + D ./ D - D .* D - D ./ D;
end
sum(E(:))
wait(dev)
toc(timer)

```

Julia:

```julia
using CUDA, BenchmarkTools
 
 function f()
     A = CUDA.rand(151,151,151) .+ 1;
     B = CUDA.rand(151,151,151) .+ 1;
     C = CUDA.zeros(151,151,151) .+ 1;
     D = similar(C);
     E = similar(C);
     F = similar(C);
     @btime CUDA.@sync begin for iter = 1:1000
             @fastmath @. $C = $A^2 + $B^2 + $A * $B + $A / $B - $A * $B - $A / $B + $A * $B + $A / $B - $A * $B - $A / $B
             @fastmath @. $D = $C^2 + $C^2 + $C * $C + $C / $C - $C * $C - $C / $C + $C * $C + $C / $C - $C * $C - $C / $C
             @fastmath @. $E = $D^2 + $D^2 + $D * $D + $D / $D - $D * $D - $D / $D + $D * $D + $D / $D - $D * $D - $D / $D
         end
         sum($E)
     end 
 end
 
 f()

 
f()

```

---

<div class="post-metadata">

**Author:** ![Alex90](https://avatars.discourse-cdn.com/v4/letter/a/6de8d8/32.png) [@Alex90](https://discourse.julialang.org/u/Alex90)\
**Post date:** [November 15, 2024, 1:39pm UTC](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663/28 "2024-11-15T13:39:31Z")

</div>

Thank you for this test! It is interesting to note that i got the same performance as yours RTX 3060 in Google Colab with Tesla T4 GPU.  
`1.523 s (432000 allocations: 204.62 MiB)`  
Moreover, Tesla T4 allocates less than GTX 960.  
In my humble opinion, it is really strange that Julia which is developed for high performance scientific computing cannot reach Matlab speed at least. One can say that you can write a CUDA kernel in Julia. On the contrary, you can run CUDA kernel in Matlab too

---

<div class="post-metadata">

**Author:** ![ufechner7](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ufechner7/32/51363_2.png) [@ufechner7](https://discourse.julialang.org/u/ufechner7)\
**Post date:** [November 15, 2024, 1:47pm UTC](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663/29 "2024-11-15T13:47:26Z")

</div>

Feel free to create an issue, including a reproducible example at:

> **[JuliaGPU/CUDA.jl](https://github.com/JuliaGPU/CUDA.jl/issues)**
>
> CUDA programming in Julia. Contribute to JuliaGPU/CUDA.jl development by creating an account on GitHub.

Currently, there are 24 open issues with the label “Performance”: [GitHub · Where software is built](https://github.com/JuliaGPU/CUDA.jl/issues?q=is%3Aissue%20state%3Aopen%20label%3Aperformance)

---

<div class="post-metadata">

**Author:** ![roflmaostc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/roflmaostc/32/30123_2.png) [@roflmaostc](https://discourse.julialang.org/u/roflmaostc)\
**Post date:** [November 15, 2024, 1:50pm UTC](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663/30 "2024-11-15T13:50:30Z")

</div>

> [@Alex90](#):
>
> cannot reach Matlab speed at least.

I agree with you but on the other hand it took me 2min to find an example where Julia outperforms Matlab by a factor of 5 🙂

Matlab took 9.729ms whereas Julia took 1.933ms.

```matlab
A = (single(rand(1024,1024,30,'gpuArray')));
B = (single(rand(1024,1024,30,'gpuArray')));

tic
dev = gpuDevice()
C = sqrt(A) .* exp(B) .* exp(1j .* B .* A);
wait(dev)
toc

```

```julia
 using CUDA, BenchmarkTools
 
 function f(A, B)
     return sqrt.(A) .* exp.(B) .* exp.(1im .* B .* A)
 end
 
 function f()
     A = CUDA.rand(1024, 1024, 30)
     B = CUDA.rand(1024, 1024, 30)
     @btime CUDA.@sync f($A, $B)
     return 0
 end
 
 f()

```

From my playing, it seems Julia struggles a bit with the loops. Maybe there is some unnecessary synchronization which happens at each loop?

---

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [November 15, 2024, 1:52pm UTC](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663/31 "2024-11-15T13:52:50Z")

</div>

> [@roflmaostc](#):
>
> One minor mistake is to [not wait for the device in Matlab](https://ch.mathworks.com/matlabcentral/answers/538991-confusing-results-when-measuring-time-of-function-on-cpu-gpu-with-tic-toc-gpu-timeit#answer_444111).

@roflmaostc Can you check what happens to the Matlab performance if you wait for the device _inside_ the loop? i.e.

```julia
for iter = 1:1000
    % disp(iter)
    C = A.^2 + B.^2 + A .* B + A ./ B - A .* B - A ./ B + A .* B + A ./ B - A .* B - A ./ B;
    D = C.^2 + C.^2 + C .* C + C ./ C - C .* C - C ./ C + C .* C + C ./ C - C .* C - C ./ C;
    E = D.^2 + D.^2 + D .* D + D ./ D - D .* D - D ./ D + D .* D + D ./ D - D .* D - D ./ D;
    wait(dev)
end

```

or maybe even

```julia
for iter = 1:1000
    % disp(iter)
    C = A.^2 + B.^2 + A .* B + A ./ B - A .* B - A ./ B + A .* B + A ./ B - A .* B - A ./ B;
    wait(dev)
    D = C.^2 + C.^2 + C .* C + C ./ C - C .* C - C ./ C + C .* C + C ./ C - C .* C - C ./ C;
    wait(dev)
    E = D.^2 + D.^2 + D .* D + D ./ D - D .* D - D ./ D + D .* D + D ./ D - D .* D - D ./ D;
    wait(dev)
end

```

?

~~In julia, these broadcast operations wait after each operation, so I’m curious if that’s the difference.~~

---

<div class="post-metadata">

**Author:** ![roflmaostc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/roflmaostc/32/30123_2.png) [@roflmaostc](https://discourse.julialang.org/u/roflmaostc)\
**Post date:** [November 15, 2024, 1:55pm UTC](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663/32 "2024-11-15T13:55:41Z")

</div>

> [@Mason](#):
>
> ```julia
> for iter = 1:1000
> % disp(iter)
> C = A.^2 + B.^2 + A .* B + A ./ B - A .* B - A ./ B + A .* B + A ./ B - A .* B - A ./ B;
> D = C.^2 + C.^2 + C .* C + C ./ C - C .* C - C ./ C + C .* C + C ./ C - C .* C - C ./ C;
> E = D.^2 + D.^2 + D .* D + D ./ D - D .* D - D ./ D + D .* D + D ./ D - D .* D - D ./ D;
> wait(dev)
> end
> 
> ```

0.942s

> [@Mason](#):
>
> ```julia
> for iter = 1:1000
> % disp(iter)
> C = A.^2 + B.^2 + A .* B + A ./ B - A .* B - A ./ B + A .* B + A ./ B - A .* B - A ./ B;
> wait(dev)
> D = C.^2 + C.^2 + C .* C + C ./ C - C .* C - C ./ C + C .* C + C ./ C - C .* C - C ./ C;
> wait(dev)
> E = D.^2 + D.^2 + D .* D + D ./ D - D .* D - D ./ D + D .* D + D ./ D - D .* D - D ./ D;
> wait(dev)
> end
> 
> ```

1.361s

---

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [November 15, 2024, 1:57pm UTC](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663/33 "2024-11-15T13:57:11Z")

</div>

Yeah, that seems to more or less account for the difference then.

---

<div class="post-metadata">

**Author:** ![roflmaostc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/roflmaostc/32/30123_2.png) [@roflmaostc](https://discourse.julialang.org/u/roflmaostc)\
**Post date:** [November 15, 2024, 1:58pm UTC](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663/34 "2024-11-15T13:58:50Z")

</div>

Is there a way to fix it?  
Is the synchronization needed? I guess not? Because the statements are still worked through sequentially?

Maybe even some sort `CUDA.@no_sync` to avoid synchronization in a block?

---

<div class="post-metadata">

**Author:** ![Alex90](https://avatars.discourse-cdn.com/v4/letter/a/6de8d8/32.png) [@Alex90](https://discourse.julialang.org/u/Alex90)\
**Post date:** [November 15, 2024, 2:02pm UTC](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663/35 "2024-11-15T14:02:42Z")

</div>

I got the same result when comparing short formulas calculations in Julia and Matlab. Julia GPU was significantly faster than Matlab GPU. That’s why i rewrote my Matlab code in Julia. But my Matlab code contains many vectorized operations like in the current example. And it appears that such style of code writing is a bottleneck for Julia

---

<div class="post-metadata">

**Author:** ![roflmaostc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/roflmaostc/32/30123_2.png) [@roflmaostc](https://discourse.julialang.org/u/roflmaostc)\
**Post date:** [November 15, 2024, 2:07pm UTC](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663/36 "2024-11-15T14:07:20Z")

</div>

You could consider sharing another code snippet in a new thread of more code.

Maybe there are some low hanging fruits as in your initial post. Translating from Matlab to Julia involves sometimes some subtle changes to speed it up.

---

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [November 15, 2024, 2:19pm UTC](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663/37 "2024-11-15T14:19:52Z")

</div>

> [@Mason](#):
>
> Yeah, that seems to more or less account for the difference then.

Hm, no apparently broadcast calls in julia are _not_ synchronizing, so this is confusing me even more. Apparently only memory copies synchronize here.

---

<div class="post-metadata">

**Author:** ![Alex90](https://avatars.discourse-cdn.com/v4/letter/a/6de8d8/32.png) [@Alex90](https://discourse.julialang.org/u/Alex90)\
**Post date:** [November 15, 2024, 2:21pm UTC](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663/38 "2024-11-15T14:21:00Z")

</div>

For now, i cannot do it since this code is a part of my research work. But the basic idea is the same as in the current example. I have around 180 lines of vectorzied operations (average length formulas). I tried to combine some formulas and reduced the number of lines more than 3 times making a very long formulas like in the present example. It was worsen the performance. Then, i tried to rewrite the code using short formulas and doubled the number of the lines. The performance was the same as in the case with average length formulas

---

<div class="post-metadata">

**Author:** ![roflmaostc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/roflmaostc/32/30123_2.png) [@roflmaostc](https://discourse.julialang.org/u/roflmaostc)\
**Post date:** [November 15, 2024, 2:24pm UTC](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663/39 "2024-11-15T14:24:42Z")

</div>

> [@lmiq](#):
>
> ```julia
> julia> @time math1!(C,A,B);
> 0.000845 seconds (11 allocations: 352 bytes)
> 
> ```
> 
> after:
> 
> ```julia
> julia> @time math1!(C,A,B);
> 0.000966 seconds
> 
> ```
> 
> (this was with a smaller matrix, just for testing)

I think splitting the long lines over different lines might reduce allocations but the runtime was the same, even for our small examples here, right?

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [November 15, 2024, 3:06pm UTC](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663/40 "2024-11-15T15:06:23Z")

</div>

Yes, but I didn’t know the effect of that when the code runs on the GPU.

[Previous page](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663.md?page=1)

[Next page](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663.md?page=3)