# How to move an array of struct with array members to GPU?

**URL:** <https://discourse.julialang.org/t/how-to-move-an-array-of-struct-with-array-members-to-gpu/110240>\
**Category:** GPU\
**Created:** [February 15, 2024, 7:54am UTC](https://discourse.julialang.org/t/how-to-move-an-array-of-struct-with-array-members-to-gpu/110240 "2024-02-15T07:54:59Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![zhzhang](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/zhzhang/32/218317_2.png) [@zhzhang](https://discourse.julialang.org/u/zhzhang)\
**Post date:** [February 15, 2024, 7:54am UTC](https://discourse.julialang.org/t/how-to-move-an-array-of-struct-with-array-members-to-gpu/110240/1 "2024-02-15T07:54:59Z")

</div>

Hello,

Recently I wanted to write a finite volume method navier-stokes solver with CUDA.jl. I was planning to store my mesh as a large array of `cell` structs, which stores positions, indices, primitive variables, etc for each cell. The MWE is as follows,

```julia
using CUDA,Adapt,StaticArrays
struct Point3D{T} <: FieldVector{3, T}
    x::T
    y::T
    z::T
end
struct cell
    a::Int
    b::AbstractArray{Int64,1}
    c::Point3D{Int64}
end

Adapt.@adapt_structure cell
grid = [cell(i, [i+2, i+3], Point3D(i+3, i+4, i+5)) for i=1:10]
grid = Adapt.adapt.(CuArray, grid) # adapt cell?
grid = Adapt.adapt(CuArray, grid) # adapt grid?

function foo(node::cell)
    node.b[2]=node.a*node.b[1]+node.c.y
end
foo.(grid)

```

I used Adapt.jl hoping to transfer the `cell` struct to a CUDA-compatible type, and the `grid` should also be a `CuArray{cell,1}`.

When I tried to broadcast the `grid` to some simple functions, it prompted an error as `ERROR: CuArray only supports element types that are allocated inline. Any is not allocated inline`. I suspected the adapted data is still incompatible with GPU but I checked the types of `grid` and `grid[1].b`, they are both `CuArray`.

So here is my question, can I move an array of struct with array members to GPU, and if it is possible, how to do this?

I am quite new to CUDA.jl and the finite volume method, if such struct is not workable for broadcasting on GPU, I may try other data structures to store my grid and cells.

Thank you for your help!  
Best regards.

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [February 15, 2024, 8:20am UTC](https://discourse.julialang.org/t/how-to-move-an-array-of-struct-with-array-members-to-gpu/110240/2 "2024-02-15T08:20:23Z")

</div>

> [@zhzhang](#):
>
> can I move an array of struct with array members to GPU

Not easily. The problem is that when launching a kernel, we convert CuArray to CuDeviceArray. When passing objects that contain CuArrays (e.g. your `grid` with `cell`s here), we would need to allocate new objects containing CuDeviceArrays, which is generally unacceptably expensive for a kernel launch. If you really want to do this, there’s a couple of hacky solutions to be found on Discourse here, which either boil down to you manually putting `CuDeviceArray`s in your structures, or using a more complicated adaptor like [Arrays of arrays and arrays of structures in CUDA kernels cause random errors - #21 by maleadt](https://discourse.julialang.org/t/arrays-of-arrays-and-arrays-of-structures-in-cuda-kernels-cause-random-errors/69739/21)

Also note that for `cell` to be GPU compatible, it cannot contain an `AbstractArray`, but the `b` type should be parametric.

---

<div class="post-metadata">

**Author:** ![zhzhang](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/zhzhang/32/218317_2.png) [@zhzhang](https://discourse.julialang.org/u/zhzhang)\
**Post date:** [February 15, 2024, 9:45am UTC](https://discourse.julialang.org/t/how-to-move-an-array-of-struct-with-array-members-to-gpu/110240/3 "2024-02-15T09:45:30Z")

</div>

Thank you for clarifying my question! It seems quite complex to hack this. I think it would be better to reconsider my data structure 🤔

> [@maleadt](#):
>
> Also note that for `cell` to be GPU compatible, it cannot contain an `AbstractArray`, but the `b` type should be parametric.

Thank you again! I did not know this before.