# GPU Julia vs GPU Matlab

**URL:** <https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663>\
**Category:** New to Julia\
**Tags:** gpu\
**Created:** [November 15, 2024, 5:14am UTC](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663 "2024-11-15T05:14:56Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![Alex90](https://avatars.discourse-cdn.com/v4/letter/a/6de8d8/32.png) [@Alex90](https://discourse.julialang.org/u/Alex90)\
**Post date:** [November 15, 2024, 5:14am UTC](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663/1 "2024-11-15T05:14:56Z")

</div>

Hello,  
I have tested a simple math example in Julia using GPU computing and GTX 960. The code is as follows:

```julia
A = CUDA.rand(151,151,151);
B = similar(A);
C = CUDA.zeros(151,151,151);
D = similar(C);
E = similar(C);
F = similar(C);
@. function math1(A,B)
    C = A.^2 .+ B.^2 .+ A .* B .+ A ./ B .- A .* B .- A ./ B .+ A .* B .+ A ./ B .- A .* B .- A ./ B;
    return C
end
@. function math2(C)
    D = C.^2 .+ C.^2 .+ C .* C .+ C ./ C .- C .* C .- C ./ C .+ C .* C .+ C ./ C .- C .* C .- C ./ C;
    return D
end
@. function math3(D)
    E= D.^2 .+ D.^2 .+ D .* D .+ D ./ D .- D .* D .- D ./ D .+ D .* D .+ D ./ D .- D .* D .- D ./ D;
    return E
end
@btime for iter = 1:1000
    C = math1(A,B);
    D = math2(C);
    E = math3(D)
end
10.619 s (1671000 allocations: 26.09 MiB)

```

And i run the same code in Matlab(2023b) and got around 2.7 seconds.  
A similar performance problem was discussed in [Why Julia is much slower than MATLAB on GPU computing?](https://discourse.julialang.org/t/why-julia-is-much-slower-than-matlab-on-gpu-computing/106450)  
So i am curious is it expectable performance behavior? Can Julia be faster than Matlab without writing a GPU kernel manually?

---

<div class="post-metadata">

**Author:** ![danielwe](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/danielwe/32/35657_2.png) [@danielwe](https://discourse.julialang.org/u/danielwe)\
**Post date:** [November 15, 2024, 5:41am UTC](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663/2 "2024-11-15T05:41:27Z")

</div>

Here’s a more idiomatic Julia version:

```julia
A = CUDA.rand(151,151,151);
B = similar(A);
C = CUDA.zeros(151,151,151);
D = similar(C);
E = similar(C);
F = similar(C);
function math1!(C, A, B)
    @. C = A^2 + B^2 + A * B + A / B - A * B - A / B + A * B + A / B - A * B - A / B
    return C
end
function math2!(D, C)
    @. D = C^2 + C^2 + C * C + C / C - C * C - C / C + C * C + C / C - C * C - C / C
    return D
end
function math3!(E, D)
    @. E = D^2 + D^2 + D * D + D / D - D * D - D / D + D * D + D / D - D * D - D / D
    return E
end
@btime for iter = 1:1000
    math1!($C, $A, $B)
    math2!($D, $C)
    math3!($E, $D)
end

```

What time do you get with this code?

Key points:

- If you want to mutate your preallocated arrays `C, D, E`, you need to pass them to the functions as arguments, and apply a mutating operation to them. `C = ...` does not mutate, it creates a new variable `C`. However, `@. C = ...` mutates because it translates to `C .= ...`, which is syntax for in-place broadcasting.
- The macro `@.` goes in front of an expression to insert dots (i.e., fused broadcasting) at every subexpression. When you use it, you don’t have to insert any dots manually. (I’ve never seen it placed in front of the entire method definition before, not sure what it would do there.)
- When benchmarking with `@btime` you should interpolate global variables with `$`, otherwise the expression can’t be type inferred and compiled to efficient code.

Minor point: you don’t need to terminate lines with a semicolon in Julia.

Let us know how the updated code performs!

---

<div class="post-metadata">

**Author:** ![Alex90](https://avatars.discourse-cdn.com/v4/letter/a/6de8d8/32.png) [@Alex90](https://discourse.julialang.org/u/Alex90)\
**Post date:** [November 15, 2024, 6:21am UTC](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663/3 "2024-11-15T06:21:05Z")

</div>

Dear Daniel,  
Many thanks for your response. I tried a very similar code modification before and it gave me the same result. On the other hand, the code revised by you provides this output:  
`10.512 s (1698000 allocations: 26.64 MiB)`  
As i understand, Julia does’t like long formulas. Previously, i did another performance test where i did no more than 3 operations (sum and double square). In this case, Julia significantly outperformed Matlab

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [November 15, 2024, 6:39am UTC](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663/4 "2024-11-15T06:39:49Z")

</div>

Is it possible that dot fusion gives up after too many operations?

---

<div class="post-metadata">

**Author:** ![rveltz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rveltz/32/2707_2.png) [@rveltz](https://discourse.julialang.org/u/rveltz)\
**Post date:** [November 15, 2024, 6:44am UTC](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663/5 "2024-11-15T06:44:14Z")

</div>

Try putting the whole computation in a function to which you pass A B C DCE

---

<div class="post-metadata">

**Author:** ![Alex90](https://avatars.discourse-cdn.com/v4/letter/a/6de8d8/32.png) [@Alex90](https://discourse.julialang.org/u/Alex90)\
**Post date:** [November 15, 2024, 6:56am UTC](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663/6 "2024-11-15T06:56:25Z")

</div>

Dear rveltz,  
I have tried and it gave the same result

---

<div class="post-metadata">

**Author:** ![rveltz](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rveltz/32/2707_2.png) [@rveltz](https://discourse.julialang.org/u/rveltz)\
**Post date:** [November 15, 2024, 7:10am UTC](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663/7 "2024-11-15T07:10:56Z")

</div>

> [@Alex90](#):
>
> `C ./ C`

You start with zeros and then NaN right? Not sure how the GPU handles that

---

<div class="post-metadata">

**Author:** ![danielwe](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/danielwe/32/35657_2.png) [@danielwe](https://discourse.julialang.org/u/danielwe)\
**Post date:** [November 15, 2024, 7:14am UTC](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663/8 "2024-11-15T07:14:12Z")

</div>

> [@danielwe](#):
>
> ```julia
> A = CUDA.rand(151,151,151);
> B = similar(A);
> C = CUDA.zeros(151,151,151);
> D = similar(C);
> E = similar(C);
> F = similar(C);
> function math1!(C, A, B)
> @. C = A^2 + B^2 + A * B + A / B - A * B - A / B + A * B + A / B - A * B - A / B
> return C
> end
> 
> ```

@rveltz might be onto something. `B` is never initialized; `similar` returns uninitialized memory. What did you intend to fill `B` with before your computations?

---

<div class="post-metadata">

**Author:** ![danielwe](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/danielwe/32/35657_2.png) [@danielwe](https://discourse.julialang.org/u/danielwe)\
**Post date:** [November 15, 2024, 7:24am UTC](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663/9 "2024-11-15T07:24:40Z")

</div>

`B = CUDA.rand(151, 151, 151)` helped a little bit on my end, but not much. Perhaps there’s a limit to loop fusion as @gdalle suggests? Broadcast support for GPU arrays, including CuArrays, is implemented using KernelAbstractions, which I’ve never used. Here’s the relevant method: [GPUArrays.jl/src/host/broadcast.jl at 77d7d5104cd1deea1a22343ff6fad2b7d65b3c28 · JuliaGPU/GPUArrays.jl · GitHub](https://github.com/JuliaGPU/GPUArrays.jl/blob/77d7d5104cd1deea1a22343ff6fad2b7d65b3c28/src/host/broadcast.jl#L47)

---

<div class="post-metadata">

**Author:** ![Alex90](https://avatars.discourse-cdn.com/v4/letter/a/6de8d8/32.png) [@Alex90](https://discourse.julialang.org/u/Alex90)\
**Post date:** [November 15, 2024, 8:05am UTC](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663/10 "2024-11-15T08:05:37Z")

</div>

You are right. It should be  
`B = CUDA.rand(151,151,151)`  
Then, there are no more NaN values. However, it slightly affected the performance. The output is as follows:  
` 9.379 s (1671000 allocations: 26.09 MiB)`

---

<div class="post-metadata">

**Author:** ![Alex90](https://avatars.discourse-cdn.com/v4/letter/a/6de8d8/32.png) [@Alex90](https://discourse.julialang.org/u/Alex90)\
**Post date:** [November 15, 2024, 8:07am UTC](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663/11 "2024-11-15T08:07:42Z")

</div>

Thanks for the link!

---

<div class="post-metadata">

**Author:** ![sdanisch](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sdanisch/32/1406_2.png) [@sdanisch](https://discourse.julialang.org/u/sdanisch)\
**Post date:** [November 15, 2024, 8:52am UTC](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663/12 "2024-11-15T08:52:14Z")

</div>

You’re using float64 on a pretty old GPU, which is really slow. To give the GPU a chance, you should switch to float32

---

<div class="post-metadata">

**Author:** ![tamasgal](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamasgal/32/27946_2.png) [@tamasgal](https://discourse.julialang.org/u/tamasgal)\
**Post date:** [November 15, 2024, 8:57am UTC](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663/13 "2024-11-15T08:57:01Z")

</div>

I am wondering if MATLAB is doing some Float32 conversions…

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [November 15, 2024, 9:02am UTC](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663/14 "2024-11-15T09:02:33Z")

</div>

Is the code non allocating when run in the CPU?

---

<div class="post-metadata">

**Author:** ![Alex90](https://avatars.discourse-cdn.com/v4/letter/a/6de8d8/32.png) [@Alex90](https://discourse.julialang.org/u/Alex90)\
**Post date:** [November 15, 2024, 9:33am UTC](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663/15 "2024-11-15T09:33:36Z")

</div>

Dear sdanisch,  
I used Float32 both with Julia and Matlab

---

<div class="post-metadata">

**Author:** ![Alex90](https://avatars.discourse-cdn.com/v4/letter/a/6de8d8/32.png) [@Alex90](https://discourse.julialang.org/u/Alex90)\
**Post date:** [November 15, 2024, 9:35am UTC](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663/16 "2024-11-15T09:35:34Z")

</div>

Dear tamasgal,  
Before computations, i created gpuArrays in Matlab with a single precision (Float32)

---

<div class="post-metadata">

**Author:** ![Alex90](https://avatars.discourse-cdn.com/v4/letter/a/6de8d8/32.png) [@Alex90](https://discourse.julialang.org/u/Alex90)\
**Post date:** [November 15, 2024, 9:46am UTC](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663/17 "2024-11-15T09:46:26Z")

</div>

Dear Imiq,  
My version of the code run on CPU gives this output:  
`45.033 s (15000 allocations: 656.25 KiB)`  
The code revised by danielwe provides another result:  
`47.197 s (33000 allocations: 1.01 MiB)`

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [November 15, 2024, 9:48am UTC](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663/18 "2024-11-15T09:48:23Z")

</div>

Then there is an issue with broadcasting fusion. Maybe the `*` symbols are still performing matrix multiplication? I never fully trust `@.` because I’m not sure how it works

---

<div class="post-metadata">

**Author:** ![Alex90](https://avatars.discourse-cdn.com/v4/letter/a/6de8d8/32.png) [@Alex90](https://discourse.julialang.org/u/Alex90)\
**Post date:** [November 15, 2024, 9:57am UTC](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663/19 "2024-11-15T09:57:55Z")

</div>

Without `@.` i get the same results when performing element-wise operations:  
` 9.458 s (1671000 allocations: 26.09 MiB)`

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [November 15, 2024, 10:07am UTC](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663/20 "2024-11-15T10:07:17Z")

</div>

Then I would probably try to split that long line in a few smaller lines, too see what happens.

Edit: effectielly, if you break that long line, it becomes non-allocating in the CPU:

```julia
A = rand(51,51,51);
B = rand(A);
C = zeros(51,51,51);
D = similar(C);
E = similar(C);
F = similar(C);
function math1!(C, A, B)
    @. C = A^2 + B^2 + A * B 
    @. C += A / B - A * B - A / B + A * B + A / B - A * B - A / B
    return C
end

```

before:

```julia-repl
julia> @time math1!(C,A,B);
  0.000845 seconds (11 allocations: 352 bytes)

```

after:

```julia-repl
julia> @time math1!(C,A,B);
  0.000966 seconds

```

(this was with a smaller matrix, just for testing)

[Next page](https://discourse.julialang.org/t/gpu-julia-vs-gpu-matlab/122663.md?page=2)