# Casting, annotations and numeric types for CUDAnative

**URL:** <https://discourse.julialang.org/t/casting-annotations-and-numeric-types-for-cudanative/19853>\
**Category:** GPU\
**Tags:** type, parametric-types\
**Created:** [January 20, 2019, 9:22pm UTC](https://discourse.julialang.org/t/casting-annotations-and-numeric-types-for-cudanative/19853 "2019-01-20T21:22:26Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Alex\_Ellison](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/alex_ellison/32/5749_2.png) [@Alex\_Ellison](https://discourse.julialang.org/u/Alex_Ellison)\
**Post date:** [January 20, 2019, 9:22pm UTC](https://discourse.julialang.org/t/casting-annotations-and-numeric-types-for-cudanative/19853/1 "2019-01-20T21:22:26Z")

</div>

Hello,

I’ve begun exploring the CUDAnative package which modest success. I’ve run into a couple of limitations.

I wanted to allocate an array from within my kernel as is shown in some of the project examples. The first limitation appears that this must be a 1 dimensional collection. Ok, I have a 2D grid and want to perform a vector operation for each, so I need to use a linear array for what I’d ideally index as 3D. I wrote a function to do this, and it only takes in integers from the block/grid dimensions and adds/multiplies them.

I can see that CUDA is giving me the results back as floats (if I attempt to store this converted index as grid output, it will crash if the container expects Int32). I’ve attempted adding type annotations, and casting but neither of those compile.

I have a minimal working example below. My questions are:

1. Given the goal of allocating/using something like a multidimensional array, is there a better way to proceed? Are there index functions to handle this?
2. Are there ever times when I can cast/annotate types onto my variables?

Random potentially useful tidbits:  
Julia version 1.0.3  
Geforce 755m  
CUDAnative 1.0.0  
CUDAdrv 1.0.1  
CuArrays 0.9.0

Thanks,  
Alex

Example:

```julia
using CUDAdrv, CUDAnative, CuArrays

function l_0(x, y, z, w, h)
       return x + y*w + z*w*h
end

function l(x, y, z, w, h)
       _x = x - 1
       _y = y - 1
       _z = z - 1
       return l_0(_x, _y, _z, w, h) + 1
end

function kernel(out)
	x = blockIdx().x
	y = blockIdx().y
	w = gridDim().x
	h = gridDim().y
	z = 1

	# apparently @cuDynamicSharedMem can only be 1 dimensional?
	arr = @cuDynamicSharedMem(Int32, w * h * 3)
	# it allows me to say `linear_index :: Int..` but not `linear_index :: Int32..`
	linear_index = l(x,y,z,w,h)

	out[x, y] = linear_index
	return nothing
end

function make_matrix(width :: Int, height :: Int)
	grid = (width, height)
	threads = (1,)

	# I can't change this Float32 -> Int32
	cu_out = CuArray{Float32, 2}(undef, width, height)

	@cuda blocks=grid threads=threads kernel(cu_out)
	out = Array{Float32, 2}(cu_out)
	return out
end

function main()
	width = 10
	height = 10
	matrix = make_matrix(width, height)
	println(matrix)
end

main()

```

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [January 21, 2019, 9:36am UTC](https://discourse.julialang.org/t/casting-annotations-and-numeric-types-for-cudanative/19853/2 "2019-01-21T09:36:19Z")

</div>

> I can’t change this Float32 → Int32

Why not? Doing so just works…

> apparently @cuDynamicSharedMem can only be 1 dimensional?

No, it can be higher-dimensional but you need to pass the size as a tuple. `@cu...SharedMem` macros are a bit of a hack and really should be a proper array type, at which point they should probably implement all of the common constructor syntaxes.

> it allows me to say `linear_index :: Int..` but not `linear_index :: Int32..`

Again, that just works here. Do know that generally you want to avoid such patterns since it generates quite ugly code (branches for checked arithmetic, allocs and calls to exception methods) that typically is unwanted in hot code. GPU intrinsics return Int32 values but the Julia functions return Int (for codegen reasons) so you can safely `unsafe_trunc(Int32, ...)` if you really want a 32-bit value.

Maybe you forgot to reserve memory for the dynamic shared memory (specified using the `shmem` keyword argument to `@cuda`)? Do note that your use of dynamic shared memory is kind-of peculiar, since it’s normally shared between threads but you only use a single thread. Doing so will probably hurt your occupancy. If you want truly local memory, why not use StaticArrays?

Anyway, here’s the code that works:

```julia
using CUDAdrv, CUDAnative, CuArrays

function l_0(x, y, z, w, h)
       return x + y*w + z*w*h
end

function l(x, y, z, w, h)
       _x = x - 1
       _y = y - 1
       _z = z - 1
       return l_0(_x, _y, _z, w, h) + 1
end

function kernel(out)
    x = blockIdx().x
    y = blockIdx().y
    w = gridDim().x
    h = gridDim().y
    z = 1

    arr = @cuDynamicSharedMem(Int32, (w, h, 3))
    arr[x,y,z] = Int32(1)
    linear_index::Int32 = l(x,y,z,w,h)
    arr[linear_index] = Int32(1) # still works

    out[x, y] = linear_index
    return nothing
end

function make_matrix(width :: Int, height :: Int)
    grid = (width, height)
    threads = (1,)

    cu_out = CuArray{Int32, 2}(undef, width, height)

    @cuda blocks=grid threads=threads shmem=sizeof(Int32)*prod(grid) kernel(cu_out)
    out = Array{Int32, 2}(cu_out)
    return out
end

function main()
    width = 10
    height = 10
    matrix = make_matrix(width, height)
    println(matrix)
end

main()

```

---

<div class="post-metadata">

**Author:** ![Alex\_Ellison](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/alex_ellison/32/5749_2.png) [@Alex\_Ellison](https://discourse.julialang.org/u/Alex_Ellison)\
**Post date:** [January 21, 2019, 6:02pm UTC](https://discourse.julialang.org/t/casting-annotations-and-numeric-types-for-cudanative/19853/3 "2019-01-21T18:02:55Z")

</div>

Hey @maleadt thanks for the response.

Regarding the `@cu...SharedMem` macros, tuple syntax indeed works, and I had forgotten the `shmem` keyword when invoking `@cuda`. That completely frees me of the attempt at converting to a linear index - I’m sure that looked weird, especially in a MWE. Static arrays are working great for my use case.

Speaking of MWE, if I take the code snippet you provided, it actually does not work for me, I get an error about boxing ints.

> The error:  
> ERROR: LoadError: CUDAnative.InvalidIRError(CUDAnative.CompilerContext(kernel, Tuple{CuDeviceArray{Int32,2,CUDAnative.AS.Global}}, v"3.0.0", true, nothing, nothing, nothing, nothing), Tuple{String,Array{Base.StackTraces.StackFrame,1},Any}[(“call to the Julia runtime”, [throw\_inexacterror at boot.jl:567, checked\_trunc\_sint at boot.jl:589, toInt32 at boot.jl:626, Type at boot.jl:716, convert at number.jl:7, kernel at minimal\_example.jl:23], “jl\_box\_int64”)])

I’ve been able to get along just fine from here by making kernel parameters explicitly Int32, so I don’t consider this a problem. Out of curiosity, are you running a 32 bit OS that would have the output of `l_0` in your example be an Int32?

Finally, since you mentioned occupancy, how smart is CUDAnative at mapping grids to cores? On my machine CUDAnative tells me my max grid dimensions are 5 x 6 x 7 = 210. If I invoked a 2D grid of size 10 x 21, would I be getting high occupancy?

Thanks again

---

<div class="post-metadata">

**Author:** ![Alex\_Ellison](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/alex_ellison/32/5749_2.png) [@Alex\_Ellison](https://discourse.julialang.org/u/Alex_Ellison)\
**Post date:** [January 21, 2019, 6:26pm UTC](https://discourse.julialang.org/t/casting-annotations-and-numeric-types-for-cudanative/19853/4 "2019-01-21T18:26:00Z")

</div>

Also, RE occupancy, if the following property is true

```julia
julia> CUDAdrv.MAX_THREADS_PER_BLOCK
MAX_THREADS_PER_BLOCK::CUdevice_attribute = 1

```

then using more than one thread per block won’t increase my occupancy right?

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [January 21, 2019, 6:39pm UTC](https://discourse.julialang.org/t/casting-annotations-and-numeric-types-for-cudanative/19853/5 "2019-01-21T18:39:00Z")

</div>

> [@Alex\_Ellison](#):
>
> Speaking of MWE, if I take the code snippet you provided, it actually does not work for me, I get an error about boxing ints.

Oh interesting, that should work on master 🙂 but code like that is slow, and will allocate which currently leaks data, so better work with explicit conversions.

> [@Alex\_Ellison](#):
>
> how smart is CUDAnative at mapping grids to cores

CUDAnative doesn’t do anything in that regard, it strictly works at the CUDA abstraction level where you have to deal with that yourself. CuArrays has some simplistic heuristics.

> [@Alex\_Ellison](#):
>
> CUDAdrv.MAX\_THREADS\_PER\_BLOCK

That’s just the enum value. You need to query the device with that value:

```julia
julia> CUDAdrv.MAX_THREADS_PER_BLOCK
MAX_THREADS_PER_BLOCK::CUdevice_attribute = 1

julia> attribute(device(), CUDAdrv.MAX_THREADS_PER_BLOCK)
1024

```

Alternatively, you can query the max kernel count for any given kernel (since it also depends on the kernel’s register usage) using CUDAnative’s low level APIs. Check the docstring for `@cuda`.

---

<div class="post-metadata">

**Author:** ![Alex\_Ellison](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/alex_ellison/32/5749_2.png) [@Alex\_Ellison](https://discourse.julialang.org/u/Alex_Ellison)\
**Post date:** [January 21, 2019, 9:07pm UTC](https://discourse.julialang.org/t/casting-annotations-and-numeric-types-for-cudanative/19853/6 "2019-01-21T21:07:07Z")

</div>

Enum. _facepalm_ that makes so much more sense, now I don’t have to worry about all kinds of nonsense like a warp size of 10 and values which don’t add up.