# Stack overflow on cuda

**URL:** <https://discourse.julialang.org/t/stack-overflow-on-cuda/10735>\
**Category:** GPU\
**Tags:** question\
**Created:** [May 5, 2018, 7:38pm UTC](https://discourse.julialang.org/t/stack-overflow-on-cuda/10735 "2018-05-05T19:38:29Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![swethmandava](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/swethmandava/32/4090_2.png) [@swethmandava](https://discourse.julialang.org/u/swethmandava)\
**Post date:** [May 5, 2018, 7:38pm UTC](https://discourse.julialang.org/t/stack-overflow-on-cuda/10735/1 "2018-05-05T19:38:29Z")

</div>

I’m trying to write a radix sort in julia using cuda but I get a stack overflow error. I’ve got it down to a bare minimum kernel and am not sure what’s wrong with it.

```julia
function increment_with_block_sums(d_predicateScan::CuDeviceMatrix{T}, d_blockSumScan::CuDeviceMatrix{T}, numElems::Integer) where {T}
    id = blockDim().x * (blockIdx().x - 1) + threadIdx().x

    if (id <= numElems)
        d_predicateScan[id, :] = d_predicateScan[id, :] + d_blockSumScan[blockIdx().x, :]
    end
    return
    
end
function gpu_exclusive_scan(input::CuArray{T}) where {T}

    numElems, num_features = size(input)
    gridSize = trunc(Int64, ceil(numElems/blockSize))
    block_sum = CuArray{T}(gridSize, num_features)
    d_total = CuArray{T}(1, num_features)

    # @cuda (gridSize, blockSize, blockSize * num_features * sizeof(T)) partial_exclusive_scan(input, block_sum, numElems)
    # @cuda (1, blockSize, blockSize * num_features * sizeof(T)) partial_exclusive_scan(block_sum, d_total, gridSize)
    @cuda (gridSize, blockSize) increment_with_block_sums(input, block_sum, numElems)
    return
end

rows = 5
cols = 4
a = rand(Int, rows, cols)
gpu_a = CuArray(a)
gpu_exclusive_scan(gpu_a)

```

Julia Version 0.6.2  
Commit d386e40\* (2017-12-13 18:08 UTC)  
Platform Info:  
OS: Linux (x86\_64-linux-gnu)  
CPU: Intel(R) Core™ i5-7200U CPU @ 2.50GHz  
WORD\_SIZE: 64  
BLAS: libopenblas (USE64BITINT NO\_AFFINITY NEHALEM)  
LAPACK: libopenblas64\_  
LIBM: libopenlibm  
LLVM: libLLVM-3.9.1 (ORCJIT, broadwell)

---

<div class="post-metadata">

**Author:** ![vchuravy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/vchuravy/32/8_2.png) [@vchuravy](https://discourse.julialang.org/u/vchuravy)\
**Post date:** [May 5, 2018, 7:51pm UTC](https://discourse.julialang.org/t/stack-overflow-on-cuda/10735/2 "2018-05-05T19:51:48Z")

</div>

What is your `Pkg.status()`? When I run your code I get the error `UndefVarError: blockSize not defined`

---

<div class="post-metadata">

**Author:** ![swethmandava](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/swethmandava/32/4090_2.png) [@swethmandava](https://discourse.julialang.org/u/swethmandava)\
**Post date:** [May 5, 2018, 8:06pm UTC](https://discourse.julialang.org/t/stack-overflow-on-cuda/10735/3 "2018-05-05T20:06:48Z")

</div>

Sorry blockSize=1024 is a global variable defined above.  
I got it to work by changing the kernel to

```julia
function increment_with_block_sums(d_predicateScan::CuDeviceMatrix{T}, d_blockSumScan::CuDeviceMatrix{T}, numElems::Integer, num_features::Integer) where {T}
    id = blockDim().x * (blockIdx().x - 1) + threadIdx().x

    if (id <= numElems)
        for i in 1:num_features
            d_predicateScan[id, i] += d_blockSumScan[blockIdx().x, i]
        end
    end
    return
    
end

```

Thanks! I didn’t realize vector additions aren’t supported.

---

<div class="post-metadata">

**Author:** ![vchuravy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/vchuravy/32/8_2.png) [@vchuravy](https://discourse.julialang.org/u/vchuravy)\
**Post date:** [May 5, 2018, 8:18pm UTC](https://discourse.julialang.org/t/stack-overflow-on-cuda/10735/4 "2018-05-05T20:18:36Z")

</div>

Ah yes, I was suspecting that was the case. The problem is that `d_predicateScan[id, :]` is creating a copy and is thereby allocating. In Julia 0.7 I think we are support views on the GPU so that should allow you to not have the loop spelled out.

---

<div class="post-metadata">

**Author:** ![swethmandava](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/swethmandava/32/4090_2.png) [@swethmandava](https://discourse.julialang.org/u/swethmandava)\
**Post date:** [May 5, 2018, 8:42pm UTC](https://discourse.julialang.org/t/stack-overflow-on-cuda/10735/5 "2018-05-05T20:42:44Z")

</div>

Thanks for clarifying. Now I have another strange error.

```julia
function partial_exclusive_scan(d_list::CuDeviceMatrix{T}, d_block_sums::CuDeviceMatrix{T}, numElems::Integer, num_features::Integer) where {T}
    

    tid = threadIdx().x
    id = blockDim().x * (blockIdx().x - 1) + threadIdx().x

    s_block_scan = @cuDynamicSharedMem(T, blockSize * num_features)

    if (id > numElems)
        for feature_id in 1:num_features
            s_block_scan[(tid - 1) * num_features + feature_id] = 0
        end
    else
        for feature_id in 1:num_features
            s_block_scan[(tid - 1) * num_features + feature_id] = d_list[id, feature_id]
        end
    end

    sync_threads()
return
end

```

Returns multiple warnings of encountered incompatible llvm ir and an error of ERROR: LoadError: LLVM IR generated for partial\_exclusive\_scan(CUDAnative.CuDeviceArray{Int64,2,CUDAnative.AS.Global}, CUDAnative.CuDeviceArray{Int64,2,CUDAnative.AS.Global}, Int64, Int64) at capability 5.0.0 is not compatible.

I’m used to Cuda on C++ and am working on a school project to compare Julia and C++ on a machine learning problem so am fairly new to this language. Sorry if it’s trivial 🙂

---

<div class="post-metadata">

**Author:** ![swethmandava](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/swethmandava/32/4090_2.png) [@swethmandava](https://discourse.julialang.org/u/swethmandava)\
**Post date:** [May 6, 2018, 12:56am UTC](https://discourse.julialang.org/t/stack-overflow-on-cuda/10735/6 "2018-05-06T00:56:48Z")

</div>

I had to replace blockSize with blockDim().x to make it work in the kernel 🙂

---

<div class="post-metadata">

**Author:** ![vchuravy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/vchuravy/32/8_2.png) [@vchuravy](https://discourse.julialang.org/u/vchuravy)\
**Post date:** [May 6, 2018, 3:30pm UTC](https://discourse.julialang.org/t/stack-overflow-on-cuda/10735/7 "2018-05-06T15:30:06Z")

</div>

In general

```julia
LLVM IR generated for ... at capability 5.0.0 is not compatible.

```

is a sign that your kernel is using a language feature that is not supported on the GPU.  
In general we try to give you information about what language feature is used, but we have little information about where the usage is coming from.

Was `blockSize` a non-const global?

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [May 8, 2018, 7:14am UTC](https://discourse.julialang.org/t/stack-overflow-on-cuda/10735/8 "2018-05-08T07:14:29Z")

</div>

Adding to @vchuravy’s comment, getting an invalid IR error can be expected when you use badly typed or unsupported code. However, that should never lead to a stack overflow as you mention in your post title. If that does happen, please file an issue with a minimal working example (you should also include that in Discourse posts like this one, because it’s much harder to help without a working example + details on the actual error).

---

<div class="post-metadata">

**Author:** ![swethmandava](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/swethmandava/32/4090_2.png) [@swethmandava](https://discourse.julialang.org/u/swethmandava)\
**Post date:** [May 9, 2018, 6:06am UTC](https://discourse.julialang.org/t/stack-overflow-on-cuda/10735/9 "2018-05-09T06:06:57Z")

</div>

No it was a global constant. may be because it isn’t on the gpu (sent to kernel specifically) it doesn’t work?

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [May 9, 2018, 8:02am UTC](https://discourse.julialang.org/t/stack-overflow-on-cuda/10735/10 "2018-05-09T08:02:41Z")

</div>

> [@swethmandava](#):
>
> may be because it isn’t on the gpu (sent to kernel specifically) it doesn’t work

That is false.

```julia
julia> using CUDAdrv, CUDAnative

julia> const foobar = 42
42

julia> const da = CuArray([1])
1-element CuArray{Int64,1}:
 1

julia> @cuda ((a)->a[1]=foobar)(da)

julia> da
1-element CuArray{Int64,1}:
 42

```

(note that the above example uses some features that are only available in CUDAnative on Julia 0.7)

Speculation about what might have gone wrong without actual code doesn’t seem productive at this point. But feel free to open an issue if you encounter the issue again 🙂