# Problems with LinearAlgebra functions within KernelAbstractions and CUDA

**URL:** <https://discourse.julialang.org/t/problems-with-linearalgebra-functions-within-kernelabstractions-and-cuda/97566>\
**Category:** General Usage\
**Tags:** cuda, linearalgebra, kernelabstractions\
**Created:** [April 17, 2023, 9:01am UTC](https://discourse.julialang.org/t/problems-with-linearalgebra-functions-within-kernelabstractions-and-cuda/97566 "2023-04-17T09:01:27Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![weymouth](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/weymouth/32/15839_2.png) [@weymouth](https://discourse.julialang.org/u/weymouth)\
**Post date:** [April 17, 2023, 9:01am UTC](https://discourse.julialang.org/t/problems-with-linearalgebra-functions-within-kernelabstractions-and-cuda/97566/1 "2023-04-17T09:01:27Z")

</div>

I’m _almost_ done porting my solver over to GPU with the help of CUDA and KernelAbstractions. However, I keep running into problem with LinearAlgebra functions. Take this simple example

```julia
using StaticArrays
using CUDA: allowscalar,CuArray
allowscalar(false)
using KernelAbstractions
using LinearAlgebra: norm2,×
using BenchmarkTools

N = (100,100,100)
f = Array # CuArray
p = zeros(Float32,N) |> f;
u = rand(Float32,N...,3) |> f;
@kernel function kern(p,u)
    I = @index(Global,Cartesian)
    p[I] = norm2(SVector{3}(I.I) × SVector{3}(u[I,:]))
end
apply!(p,u) = kern(get_backend(p),64)(p,u,ndrange=size(p))
@btime apply!(p,u); # 16.459 ms (1000199 allocations: 76.31 MiB)

```

This gives a crazy number of allocations running on a CPU and won’t run on the GPU

```julia
Reason: unsupported call through a literal pointer (call to ijl_alloc_array_1d)
Reason: unsupported dynamic function invocation (call to print_to_string(xs...) in Base at strings/io.jl:133)
Reason: unsupported dynamic function invocation (call to dimension_mismatch_fail(SA::Type, a::AbstractArray) in StaticArrays at C:\Users\gweymouth\.julia\packages\StaticArrays\4uslg\src\convert.jl:190)

```

If I take out the SVector conversion, the allocations go way up on the CPU and the GPU throws errors about `Reason: unsupported dynamic function invocation (call to mapreduce_empty_iter(f, op, itr, ItrEltype) in Base at reduce.jl:375)`.

---

<div class="post-metadata">

**Author:** ![weymouth](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/weymouth/32/15839_2.png) [@weymouth](https://discourse.julialang.org/u/weymouth)\
**Post date:** [April 17, 2023, 9:15am UTC](https://discourse.julialang.org/t/problems-with-linearalgebra-functions-within-kernelabstractions-and-cuda/97566/2 "2023-04-17T09:15:06Z")

</div>

It seems the main issue is with `SVector{3}(u[I,:])`. Replacing this with `SA[u[I,1],u[I,2],u[I,3]]` runs and cuts most of the allocations. Can someone explain why?

---

<div class="post-metadata">

**Author:** ![LaurentPlagne](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/laurentplagne/32/10103_2.png) [@LaurentPlagne](https://discourse.julialang.org/u/LaurentPlagne)\
**Post date:** [April 17, 2023, 11:25am UTC](https://discourse.julialang.org/t/problems-with-linearalgebra-functions-within-kernelabstractions-and-cuda/97566/3 "2023-04-17T11:25:32Z")

</div>

Does a view on `u[I,:]` help ?

---

<div class="post-metadata">

**Author:** ![trahflow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/trahflow/32/30585_2.png) [@trahflow](https://discourse.julialang.org/u/trahflow)\
**Post date:** [February 6, 2024, 3:34pm UTC](https://discourse.julialang.org/t/problems-with-linearalgebra-functions-within-kernelabstractions-and-cuda/97566/4 "2024-02-06T15:34:02Z")

</div>

I know my reply comes late, but for reference I’ll answer anyway:

The issue is actually stated in the error message (last line):

> [@weymouth](#):
>
> `Reason: unsupported dynamic function invocation (call to dimension_mismatch_fail(SA::Type, a::AbstractArray) in StaticArrays at C:\Users\gweymouth\.julia\packages\StaticArrays\4uslg\src\convert.jl:190)`

You can fix this by dropping the bounds check in the SVector constructor:

```julia
v1 = SVector{3}(I.I)
u_view = @view u[I,:]
v2 = @inbounds SVector{3}(u_view)
p[I] = norm2(v1 × v2)

```

This is also the reason why `SA[u[I,1],u[I,2],u[I,3]]` works. It doesn’t do a bounds check, which would require a dynamic function call (for throwing the error).

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [February 6, 2024, 7:55pm UTC](https://discourse.julialang.org/t/problems-with-linearalgebra-functions-within-kernelabstractions-and-cuda/97566/5 "2024-02-06T19:55:19Z")

</div>

> [@weymouth](#):
>
> call to dimension\_mismatch\_fail(SA::Type, a::AbstractArray) in StaticArrays

That `@noinline` function should probably have a type signature that forces specialization, in order to make the code GPU compatible. Maybe open an issue on StaticArraytsl.jl?

---

<div class="post-metadata">

**Author:** ![trahflow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/trahflow/32/30585_2.png) [@trahflow](https://discourse.julialang.org/u/trahflow)\
**Post date:** [February 7, 2024, 3:37pm UTC](https://discourse.julialang.org/t/problems-with-linearalgebra-functions-within-kernelabstractions-and-cuda/97566/6 "2024-02-07T15:37:55Z")

</div>

> [@maleadt](#):
>
> That `@noinline` function should probably have a type signature that forces specialization, in order to make the code GPU compatible. Maybe open an issue on [StaticArraytsl.jl](https://juliahub.com/ui/Packages/StaticArraytsl)?

if I force specialization of the function `StaticArrays.dimension_mismatch_fail` on a local checkout ouf `StaticArrays`, I just get another error:

```julia
Reason: unsupported dynamic function invocation (call to print_to_string(xs...) @ Base strings/io.jl:137)
Stacktrace:
 [1] string
   @ ./strings/io.jl:189
 [2] dimension_mismatch_fail
   @ ~/julia-depots/gpu/dev/StaticArrays/src/convert.jl:196
 [3] convert
   @ ~/julia-depots/gpu/dev/StaticArrays/src/convert.jl:201
 [4] StaticArray
   @ ~/julia-depots/gpu/dev/StaticArrays/src/convert.jl:174
 [5] macro expansion
   @ ~/.julia/dev/DiffPointRasterisation/experiments/kernelabstractions.jl:13
 [6] gpu_kern
   @ ~/julia-depots/gpu/packages/KernelAbstractions/jmHRX/src/macros.jl:101
 [7] gpu_kern
   @ ./none:0

```

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [February 12, 2024, 12:06pm UTC](https://discourse.julialang.org/t/problems-with-linearalgebra-functions-within-kernelabstractions-and-cuda/97566/7 "2024-02-12T12:06:05Z")

</div>

Ah yes that’s because of the string interpolation there. We could add a quirk like we have for Base methods doing the same, [CUDA.jl/src/device/quirks.jl at master · JuliaGPU/CUDA.jl · GitHub](https://github.com/JuliaGPU/CUDA.jl/blob/master/src/device/quirks.jl), but then in an extension package for StaticArrays.jl.

---

<div class="post-metadata">

**Author:** ![trahflow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/trahflow/32/30585_2.png) [@trahflow](https://discourse.julialang.org/u/trahflow)\
**Post date:** [February 14, 2024, 12:39pm UTC](https://discourse.julialang.org/t/problems-with-linearalgebra-functions-within-kernelabstractions-and-cuda/97566/8 "2024-02-14T12:39:03Z")

</div>

Opened an issue here: [Force specialization of `dimension_mismatch_fail()` on type parameter? · Issue #1244 · JuliaArrays/StaticArrays.jl · GitHub](https://github.com/JuliaArrays/StaticArrays.jl/issues/1244)

---

<div class="post-metadata">

**Author:** ![trahflow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/trahflow/32/30585_2.png) [@trahflow](https://discourse.julialang.org/u/trahflow)\
**Post date:** [February 22, 2024, 9:53am UTC](https://discourse.julialang.org/t/problems-with-linearalgebra-functions-within-kernelabstractions-and-cuda/97566/9 "2024-02-22T09:53:13Z")

</div>

@maleadt the issue on StaticArrays side is fixed, so I wanted to add that quirk you mentioned.  
One question though: Why as an extension package? `StaticArrays` currently is a direct/strong dependency of `CUDA`. Is there a plan to make this a weak dependency, or is there something else I don’t see?

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [February 22, 2024, 9:58am UTC](https://discourse.julialang.org/t/problems-with-linearalgebra-functions-within-kernelabstractions-and-cuda/97566/10 "2024-02-22T09:58:47Z")

</div>

I would like to get rid of the dependency at some point; it’s only there to implement KernelAbstraction’s scratchpad memory.