Store CuArrays on a mutable struct?

Specific Domains GPU
1

hi,

I seem to understand that I cannot store a (reference to?) a CuArray on a struct:

mutable struct Param
	eta :: Float64
	beta  :: Float64
	alpha :: Float64
	delta :: Float64
	mu    :: Float64
	rho   :: Float64
	sigma   :: Float64
	nk    :: Int
	nz    :: Int
	tol   :: Float64
	function Param(;par=Dict())
        f=open(joinpath(dirname(@__FILE__),"..","..","..","params.json")) 
		j = JSON.parse(f)
		close(f)
    	this = new()
    	for (k,v) in j
            setfield!(this,Symbol(k),v["value"])
    	end
    	return this
	end

end

mutable struct Model 
	V       :: Matrix{Float32}   # value fun
	V0      :: Matrix{Float32}   # value fun
	G       :: Matrix{Int}   # policy fun
	G0      :: Matrix{Int}   # policy fun
	P       :: Matrix{Float32}   # transition matrix
	zgrid   :: Vector{Float32}
	kgrid   :: StepRangeLen{Float32}
	fkgrid  :: Vector{Float32}
	ydepK   :: Matrix{Float32}
	counter :: Int
	function Model(p::Param)
		this              = new()
		this.V            = zeros(Float32,p.nk,p.nz)
		this.G            = zeros(Int,p.nk,p.nz)
		this.V0           = zeros(Float32,p.nk,p.nz)
		this.G0           = zeros(Int,p.nk,p.nz)
		this.zgrid,this.P = rouwenhorst(p.rho,p.mu,p.sigma,p.nz)
		this.zgrid = exp.(this.zgrid)
		kmin              = 0.95*(((1/(p.alpha*this.zgrid[1]))*((1/p.beta)-1+p.delta))^(1/(p.alpha-1)))
		kmax              = 1.05*(((1/(p.alpha*this.zgrid[end]))*((1/p.beta)-1+p.delta))^(1/(p.alpha-1)))
		this.kgrid        = range(kmin,step = (kmax-kmin)/(p.nk-1),length = p.nk)
		this.fkgrid       = (this.kgrid).^p.alpha
		this.counter      = 0
		# output plus depreciated capital
		this.ydepK = this.fkgrid .* this.zgrid' .+ (1-p.delta).*repeat(this.kgrid,1,p.nz)
		return this
	end
end

mutable struct CuModel 
	V       :: CuMatrix{Float32}   # value fun
	V0      :: CuMatrix{Float32}   # value fun
	G       :: CuMatrix{Int}   # policy fun
	G0      :: CuMatrix{Int}   # policy fun
	P       :: CuMatrix{Float32}   # transition matrix
	zgrid   :: CuVector{Float32}
	kgrid   :: CuVector{Float32}
	fkgrid  :: CuVector{Float32}
	ydepK   :: CuMatrix{Float32}
	counter :: Int
	function CuModel(m::Model)
		this         = new()
		this.V       = CuArray(m.V)
		this.G       = CuArray(m.G)
		this.V0      = CuArray(m.V0)
		this.G0      = CuArray(m.G0)
		this.P       = CuArray(m.P)
		this.zgrid   = CuArray(m.zgrid)
		this.kgrid   = CuArray(convert(Vector{Float32},collect(m.kgrid)))
		this.counter = 0
		# output plus depreciated capital
		this.ydepK = CuArray(m.ydepK)
		return this
	end
end

julia> cum = cudaVFI.CuModel(m)
Main.cudaVFI.CuModel(Float32[Error showing value of type Main.cudaVFI.CuModel:
ERROR: getindex not defined for CuArray{Float32,2}
Stacktrace:
 [1] error(::String, ::Type) at ./error.jl:42
 [2] error_if_canonical_getindex at ./abstractarray.jl:914 [inlined]
 [3] getindex at ./abstractarray.jl:903 [inlined]
 [4] isassigned(::CuArray{Float32,2}, ::Int64, ::Int64) at ./abstractarray.jl:350
 [5] _show_nonempty(::IOContext{REPL.Terminals.TTYTerminal}, ::CuArray{Float32,2}, ::String) at ./arrayshow.jl:382
 [6] show(::IOContext{REPL.Terminals.TTYTerminal}, ::CuArray{Float32,2}) at ./arrayshow.jl:420
 [7] show_default(::IOContext{REPL.Terminals.TTYTerminal}, ::Any) at ./show.jl:333
 [8] show at ./show.jl:316 [inlined]
 [9] show(::IOContext{REPL.Terminals.TTYTerminal}, ::MIME{Symbol("text/plain")}, ::Main.cudaVFI.CuModel) at ./sysimg.jl:197
 [10] display(::REPL.REPLDisplay{REPL.LineEditREPL}, ::MIME{Symbol("text/plain")}, ::Main.cudaVFI.CuModel) at /home/floswald/git/julia/usr/share/julia/stdlib/v0.7/REPL/src/REPL.jl:130
 [11] display(::REPL.REPLDisplay{REPL.LineEditREPL}, ::Main.cudaVFI.CuModel) at /home/floswald/git/julia/usr/share/julia/stdlib/v0.7/REPL/src/REPL.jl:133
 [12] display(::Main.cudaVFI.CuModel) at ./multimedia.jl:287
 [13] #invokelatest#1 at ./essentials.jl:670 [inlined]
 [14] invokelatest at ./essentials.jl:669 [inlined]
 [15] print_response(::REPL.Terminals.TTYTerminal, ::Any, ::Nothing, ::Bool, ::Bool, ::Nothing) at /home/floswald/git/julia/usr/share/julia/stdlib/v0.7/REPL/src/REPL.jl:151
 [16] print_response(::REPL.LineEditREPL, ::Any, ::Nothing, ::Bool, ::Bool) at /home/floswald/git/julia/usr/share/julia/stdlib/v0.7/REPL/src/REPL.jl:137
 [17] (::getfield(REPL, Symbol("#do_respond#42")){Bool,getfield(REPL, Symbol("##52#61")){REPL.LineEditREPL,REPL.REPLHistoryProvider},REPL.LineEditREPL,REPL.LineEdit.Prompt})(::REPL.LineEdit.MIState, ::Base.GenericIOBuffer{Array{UInt8,1}}, ::Bool) at /home/floswald/git/julia/usr/share/julia/stdlib/v0.7/REPL/src/REPL.jl:705
 [18] top-level scope

Is there anything wrong here other than I am storing the CuArrays in the wrong way?

Recommended way to treat multiple CuArrays

Should I have

  • An Array{CuArray}
  • a Dict
  • a struct

if I want to carry around several CuArrays, or is there no way to save them in an iterable at all?

2

Why not? That error is just displaying of the CuArray gone wrong, since it doesn’t have getindex defined. At least, CUDAdrv.CuArray doesn’t, CuArrays.jl is quite a bit userfriendlier in that regard (and does have getindex defined).

The only problem with storing CuArray in a struct, is that you lose automatic conversion to CuDeviceArray upon passing that struct to a kernel function. See More general device conversions · Issue #121 · JuliaGPU/CUDAnative.jl · GitHub

3

thanks.
I can’t get CuArrays to work on julia 0.7. anything in particular I should keep in mind? I checked out the master branch.

4

https://github.com/JuliaGPU/CuArrays.jl/pull/70

Awaiting that, you can just use CUDAdrv.CuArray and ignore the display error.

5

Hi Tim, you mentioned " … you lose automatic conversion to CuDeviceArray upon passing that struct to a kernel function. …" Do I have the conversion if I pass the fields like some_kernel(A.zgrid) where A = CuModel(m) ? I guess yes… But how much does this cost?

In my project, I stored a list of CuMatrix in a struct, say STRCT.LST, and I have to pass that (read-only) list STRCT.LST quite frequently into a kernel. In this case, can I ignore the cost of “conversion” to CuDeviceArray for CuMatrix in my list STRCT.LST?

6

Yes, because the value you end up passing is a CuArray and there exists a conversion for CuArray → CuDeviceArray. Note that you can register your own conversions, by adding a cudaconvert method (and maybe check out this issue by @MikeInnes for a more generic mechanism for device conversions).

It’s just an object conversion, and doesn’t perform an actual upload as the inner CuArray already lives on the GPU. So the cost is negligible.

Yes. But if you end up passing that struct a lot, you might want to consider creating a GPU alternative counterpart (containing CuDeviceArrays) and defining a cudaconvert method.