# Type instability with CuVector inside struct

**URL:** <https://discourse.julialang.org/t/type-instability-with-cuvector-inside-struct/97476>\
**Category:** GPU\
**Created:** [April 14, 2023, 1:50pm UTC](https://discourse.julialang.org/t/type-instability-with-cuvector-inside-struct/97476 "2023-04-14T13:50:25Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![bjarthur](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bjarthur/32/9638_2.png) [@bjarthur](https://discourse.julialang.org/u/bjarthur)\
**Post date:** [April 14, 2023, 1:50pm UTC](https://discourse.julialang.org/t/type-instability-with-cuvector-inside-struct/97476/1 "2023-04-14T13:50:25Z")

</div>

this is a very simple problem, so i fear i’m making a dumb mistake. apologies if so. but i can’t figure out why the function `foo` below is type unstable when input with a struct that has a `CuVector` in it, whereas if it’s a regular `Vector` it is fine. thx.

```julia
julia> using CUDA

julia> struct Foo
           x::Vector{Float64}
       end

julia> foo = Foo(ones(3))
Foo([1.0, 1.0, 1.0])

julia> struct CuFoo
           x::CuVector{Float64}
       end

julia> cu_foo = CuFoo(ones(3))
CuFoo([1.0, 1.0, 1.0])

julia> function fun(s)
           s.x .= 0.0
       end
fun (generic function with 1 method)

julia> @code_warntype fun(foo) # everything here is type stable
MethodInstance for fun(::Foo)
  from fun(s) in Main at REPL[6]:1
Arguments
  #self#::Core.Const(fun)
  s::Foo
Body::Vector{Float64}
1 ─ %1 = Base.dotgetproperty(s, :x)::Vector{Float64}
│ %2 = Base.broadcasted(Base.identity, 0.0)::Core.Const(Base.Broadcast.Broadcasted(identity, (0.0,)))
│ %3 = Base.materialize!(%1, %2)::Vector{Float64}
└── return %3

julia> @code_warntype fun(cu_foo) # this is NOT type stable :(
MethodInstance for fun(::CuFoo)
  from fun(s) in Main at REPL[6]:1
Arguments
  #self#::Core.Const(fun)
  s::CuFoo
Body::CuArray{Float64, 1} ### RED RED RED
1 ─ %1 = Base.dotgetproperty(s, :x)::CuArray{Float64, 1} ### RED RED RED
│ %2 = Base.broadcasted(Base.identity, 0.0)::Core.Const(Base.Broadcast.Broadcasted(identity, (0.0,)))
│ %3 = Base.materialize!(%1, %2)::CuArray{Float64, 1} ### RED RED RED
└── return %3

julia> using MethodAnalysis

julia> methodinstances(fun)
2-element Vector{Core.MethodInstance}:
 MethodInstance for fun(::Foo)
 MethodInstance for fun(::CuFoo)

```

this is with julia 1.8.5 and CUDA.jl v4.1.4.

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [April 14, 2023, 2:10pm UTC](https://discourse.julialang.org/t/type-instability-with-cuvector-inside-struct/97476/2 "2023-04-14T14:10:04Z")

</div>

CuVector has additional type variables. But it’s better to parameterize the field so that you’re not tied to the exact definition.

---

<div class="post-metadata">

**Author:** ![bjarthur](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bjarthur/32/9638_2.png) [@bjarthur](https://discourse.julialang.org/u/bjarthur)\
**Post date:** [April 14, 2023, 5:45pm UTC](https://discourse.julialang.org/t/type-instability-with-cuvector-inside-struct/97476/3 "2023-04-14T17:45:38Z")

</div>

oh right! thanks so much.

as a note to my future self, the extra parameter specifies the storage:

```julia
julia> CUDA.ones(3)
3-element CuArray{Float32, 1, CUDA.Mem.DeviceBuffer}:
 1.0
 1.0
 1.0

help?> CuArray
search: CuArray CuArrayPtr AnyCuArray DenseCuArray StridedCuArray CuDeviceArray CuTextureArray

  No documentation found.

  Summary
  ≡≡≡≡≡≡≡≡≡

  mutable struct CuArray{T, N, B}

  Fields
  ≡≡≡≡≡≡≡≡

  storage :: Union{Nothing, CUDA.ArrayStorage{B}}
  maxsize :: Int64
  offset :: Int64
  dims :: Tuple{Vararg{Int64, N}}

  Supertype Hierarchy
  ≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡

  CuArray{T, N, B} <: GPUArraysCore.AbstractGPUArray{T, N} <: DenseArray{T, N} <: AbstractArray{T, N} <: Any

```

so my above script could be re-written as:

```julia
julia> using CUDA

julia> struct Foo{T}
           x::T
       end

julia> foo = Foo(ones(3))
Foo{Vector{Float64}}([1.0, 1.0, 1.0])

julia> cu_foo = Foo(CUDA.ones(3))
Foo{CuArray{Float32, 1, CUDA.Mem.DeviceBuffer}}(Float32[1.0, 1.0, 1.0])

julia> function fun(s)
           s.x .= 0.0
       end
fun (generic function with 1 method)

julia> @code_warntype fun(foo)
MethodInstance for fun(::Foo{Vector{Float64}})
  from fun(s) in Main at REPL[5]:1
Arguments
  #self#::Core.Const(fun)
  s::Foo{Vector{Float64}}
Body::Vector{Float64}
1 ─ %1 = Base.dotgetproperty(s, :x)::Vector{Float64}
│ %2 = Base.broadcasted(Base.identity, 0.0)::Core.Const(Base.Broadcast.Broadcasted(identity, (0.0,)))
│ %3 = Base.materialize!(%1, %2)::Vector{Float64}
└── return %3

julia> @code_warntype fun(cu_foo)
MethodInstance for fun(::Foo{CuArray{Float32, 1, CUDA.Mem.DeviceBuffer}})
  from fun(s) in Main at REPL[5]:1
Arguments
  #self#::Core.Const(fun)
  s::Foo{CuArray{Float32, 1, CUDA.Mem.DeviceBuffer}}
Body::CuArray{Float32, 1, CUDA.Mem.DeviceBuffer}
1 ─ %1 = Base.dotgetproperty(s, :x)::CuArray{Float32, 1, CUDA.Mem.DeviceBuffer}
│ %2 = Base.broadcasted(Base.identity, 0.0)::Core.Const(Base.Broadcast.Broadcasted(identity, (0.0,)))
│ %3 = Base.materialize!(%1, %2)::CuArray{Float32, 1, CUDA.Mem.DeviceBuffer}
└── return %3

```

in my particular use case, i wanted to supply the struct with defaults using `Base.@kwdef`. not sure how to get around using `CUDA.Mem.DeviceBuffer` explicitly, but this works for me:

```julia
julia> N=3
3

julia> Base.@kwdef struct Bar{T}
           x::T = Vector(undef, N)
       end

julia> bar = Bar{Vector{Float64}}()
Bar{Vector{Float64}}([0.0, 1.63e-322, 0.0])

julia> cu_bar = Bar{CuVector{Float64, CUDA.Mem.DeviceBuffer}}()
Bar{CuArray{Float64, 1, CUDA.Mem.DeviceBuffer}}([-8.826581242931566e212, 4.94e-322, 0.0])

```