# Store CuArrays on a mutable struct?

**URL:** <https://discourse.julialang.org/t/store-cuarrays-on-a-mutable-struct/11605>\
**Category:** GPU\
**Created:** [June 12, 2018, 12:14pm UTC](https://discourse.julialang.org/t/store-cuarrays-on-a-mutable-struct/11605 "2018-06-12T12:14:51Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![floswald](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/floswald/32/195_2.png) [@floswald](https://discourse.julialang.org/u/floswald)\
**Post date:** [June 12, 2018, 12:14pm UTC](https://discourse.julialang.org/t/store-cuarrays-on-a-mutable-struct/11605/1 "2018-06-12T12:14:51Z")

</div>

hi,

I seem to understand that I cannot store a (reference to?) a `CuArray` on a `struct`:

```julia-auto
mutable struct Param
	eta :: Float64
	beta :: Float64
	alpha :: Float64
	delta :: Float64
	mu :: Float64
	rho :: Float64
	sigma :: Float64
	nk :: Int
	nz :: Int
	tol :: Float64
	function Param(;par=Dict())
        f=open(joinpath(dirname(@ __FILE__ ),"..","..","..","params.json")) 
		j = JSON.parse(f)
		close(f)
    	this = new()
    	for (k,v) in j
            setfield!(this,Symbol(k),v["value"])
    	end
    	return this
	end

end

mutable struct Model 
	V :: Matrix{Float32} # value fun
	V0 :: Matrix{Float32} # value fun
	G :: Matrix{Int} # policy fun
	G0 :: Matrix{Int} # policy fun
	P :: Matrix{Float32} # transition matrix
	zgrid :: Vector{Float32}
	kgrid :: StepRangeLen{Float32}
	fkgrid :: Vector{Float32}
	ydepK :: Matrix{Float32}
	counter :: Int
	function Model(p::Param)
		this = new()
		this.V = zeros(Float32,p.nk,p.nz)
		this.G = zeros(Int,p.nk,p.nz)
		this.V0 = zeros(Float32,p.nk,p.nz)
		this.G0 = zeros(Int,p.nk,p.nz)
		this.zgrid,this.P = rouwenhorst(p.rho,p.mu,p.sigma,p.nz)
		this.zgrid = exp.(this.zgrid)
		kmin = 0.95*(((1/(p.alpha*this.zgrid[1]))*((1/p.beta)-1+p.delta))^(1/(p.alpha-1)))
		kmax = 1.05*(((1/(p.alpha*this.zgrid[end]))*((1/p.beta)-1+p.delta))^(1/(p.alpha-1)))
		this.kgrid = range(kmin,step = (kmax-kmin)/(p.nk-1),length = p.nk)
		this.fkgrid = (this.kgrid).^p.alpha
		this.counter = 0
		# output plus depreciated capital
		this.ydepK = this.fkgrid .* this.zgrid' .+ (1-p.delta).*repeat(this.kgrid,1,p.nz)
		return this
	end
end

mutable struct CuModel 
	V :: CuMatrix{Float32} # value fun
	V0 :: CuMatrix{Float32} # value fun
	G :: CuMatrix{Int} # policy fun
	G0 :: CuMatrix{Int} # policy fun
	P :: CuMatrix{Float32} # transition matrix
	zgrid :: CuVector{Float32}
	kgrid :: CuVector{Float32}
	fkgrid :: CuVector{Float32}
	ydepK :: CuMatrix{Float32}
	counter :: Int
	function CuModel(m::Model)
		this = new()
		this.V = CuArray(m.V)
		this.G = CuArray(m.G)
		this.V0 = CuArray(m.V0)
		this.G0 = CuArray(m.G0)
		this.P = CuArray(m.P)
		this.zgrid = CuArray(m.zgrid)
		this.kgrid = CuArray(convert(Vector{Float32},collect(m.kgrid)))
		this.counter = 0
		# output plus depreciated capital
		this.ydepK = CuArray(m.ydepK)
		return this
	end
end

julia> cum = cudaVFI.CuModel(m)
Main.cudaVFI.CuModel(Float32[Error showing value of type Main.cudaVFI.CuModel:
ERROR: getindex not defined for CuArray{Float32,2}
Stacktrace:
 [1] error(::String, ::Type) at ./error.jl:42
 [2] error_if_canonical_getindex at ./abstractarray.jl:914 [inlined]
 [3] getindex at ./abstractarray.jl:903 [inlined]
 [4] isassigned(::CuArray{Float32,2}, ::Int64, ::Int64) at ./abstractarray.jl:350
 [5] _show_nonempty(::IOContext{REPL.Terminals.TTYTerminal}, ::CuArray{Float32,2}, ::String) at ./arrayshow.jl:382
 [6] show(::IOContext{REPL.Terminals.TTYTerminal}, ::CuArray{Float32,2}) at ./arrayshow.jl:420
 [7] show_default(::IOContext{REPL.Terminals.TTYTerminal}, ::Any) at ./show.jl:333
 [8] show at ./show.jl:316 [inlined]
 [9] show(::IOContext{REPL.Terminals.TTYTerminal}, ::MIME{Symbol("text/plain")}, ::Main.cudaVFI.CuModel) at ./sysimg.jl:197
 [10] display(::REPL.REPLDisplay{REPL.LineEditREPL}, ::MIME{Symbol("text/plain")}, ::Main.cudaVFI.CuModel) at /home/floswald/git/julia/usr/share/julia/stdlib/v0.7/REPL/src/REPL.jl:130
 [11] display(::REPL.REPLDisplay{REPL.LineEditREPL}, ::Main.cudaVFI.CuModel) at /home/floswald/git/julia/usr/share/julia/stdlib/v0.7/REPL/src/REPL.jl:133
 [12] display(::Main.cudaVFI.CuModel) at ./multimedia.jl:287
 [13] #invokelatest#1 at ./essentials.jl:670 [inlined]
 [14] invokelatest at ./essentials.jl:669 [inlined]
 [15] print_response(::REPL.Terminals.TTYTerminal, ::Any, ::Nothing, ::Bool, ::Bool, ::Nothing) at /home/floswald/git/julia/usr/share/julia/stdlib/v0.7/REPL/src/REPL.jl:151
 [16] print_response(::REPL.LineEditREPL, ::Any, ::Nothing, ::Bool, ::Bool) at /home/floswald/git/julia/usr/share/julia/stdlib/v0.7/REPL/src/REPL.jl:137
 [17] (::getfield(REPL, Symbol("#do_respond#42")){Bool,getfield(REPL, Symbol("##52#61")){REPL.LineEditREPL,REPL.REPLHistoryProvider},REPL.LineEditREPL,REPL.LineEdit.Prompt})(::REPL.LineEdit.MIState, ::Base.GenericIOBuffer{Array{UInt8,1}}, ::Bool) at /home/floswald/git/julia/usr/share/julia/stdlib/v0.7/REPL/src/REPL.jl:705
 [18] top-level scope

```

Is there anything wrong here other than I am storing the `CuArrays` in the wrong way?

### Recommended way to treat multiple `CuArrays`

Should I have

- An `Array{CuArray}`
- a `Dict`
- a `struct`

if I want to carry around several `CuArrays`, or is there no way to save them in an iterable at all?

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [June 12, 2018, 1:02pm UTC](https://discourse.julialang.org/t/store-cuarrays-on-a-mutable-struct/11605/2 "2018-06-12T13:02:33Z")

</div>

> [@floswald](#):
>
> I seem to understand that I cannot store a (reference to?) a `CuArray` on a `struct` :

Why not? That error is just displaying of the `CuArray` gone wrong, since it doesn’t have `getindex` defined. At least, `CUDAdrv.CuArray` doesn’t, CuArrays.jl is quite a bit userfriendlier in that regard (and does have `getindex` defined).

The only problem with storing `CuArray` in a struct, is that you lose automatic conversion to `CuDeviceArray` upon passing that struct to a kernel function. See [More general device conversions · Issue #121 · JuliaGPU/CUDAnative.jl · GitHub](https://github.com/JuliaGPU/CUDAnative.jl/issues/121)

---

<div class="post-metadata">

**Author:** ![floswald](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/floswald/32/195_2.png) [@floswald](https://discourse.julialang.org/u/floswald)\
**Post date:** [June 12, 2018, 1:50pm UTC](https://discourse.julialang.org/t/store-cuarrays-on-a-mutable-struct/11605/3 "2018-06-12T13:50:43Z")

</div>

thanks.  
I can’t get `CuArrays` to work on julia 0.7. anything in particular I should keep in mind? I checked out the master branch.

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [June 12, 2018, 2:24pm UTC](https://discourse.julialang.org/t/store-cuarrays-on-a-mutable-struct/11605/4 "2018-06-12T14:24:37Z")

</div>

[https://github.com/JuliaGPU/CuArrays.jl/pull/70](https://github.com/JuliaGPU/CuArrays.jl/pull/70)

Awaiting that, you can just use CUDAdrv.CuArray and ignore the display error.

---

<div class="post-metadata">

**Author:** ![Lian\_Yunlong](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lian_yunlong/32/14437_2.png) [@Lian\_Yunlong](https://discourse.julialang.org/u/Lian_Yunlong)\
**Post date:** [July 2, 2018, 5:29pm UTC](https://discourse.julialang.org/t/store-cuarrays-on-a-mutable-struct/11605/5 "2018-07-02T17:29:22Z")

</div>

Hi Tim, you mentioned " … you lose automatic conversion to `CuDeviceArray` upon passing that struct to a kernel function. …" Do I have the conversion if I pass the fields like `some_kernel(A.zgrid)` where `A = CuModel(m)` ? I guess yes… But how much does this cost?

In my project, I stored a list of CuMatrix in a struct, say `STRCT.LST`, and I have to pass that (read-only) list `STRCT.LST` quite frequently into a kernel. In this case, can I ignore the cost of “conversion” to `CuDeviceArray` for `CuMatrix` in my list `STRCT.LST`?

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [July 2, 2018, 8:17pm UTC](https://discourse.julialang.org/t/store-cuarrays-on-a-mutable-struct/11605/6 "2018-07-02T20:17:34Z")

</div>

> [@Lian\_Yunlong](#):
>
> Do I have the conversion if I pass the fields like `some_kernel(A.zgrid)` where `A = CuModel(m)`

Yes, because the value you end up passing is a `CuArray` and there exists a conversion for `CuArray` → `CuDeviceArray`. Note that you can register your own conversions, by adding a `cudaconvert` method (and maybe check out [this](https://github.com/JuliaGPU/CUDAnative.jl/issues/121) issue by @MikeInnes for a more generic mechanism for device conversions).

> [@Lian\_Yunlong](#):
>
> But how much does this cost?

It’s just an object conversion, and doesn’t perform an actual upload as the inner `CuArray` already lives on the GPU. So the cost is negligible.

> [@Lian\_Yunlong](#):
>
> In this case, can I ignore the cost of “conversion” to `CuDeviceArray` for `CuMatrix` in my list `STRCT.LST` ?

Yes. But if you end up passing that struct a lot, you might want to consider creating a GPU alternative counterpart (containing CuDeviceArrays) and defining a `cudaconvert` method.