# How to call cuda functions that are not implemented in CUDA.jl?

**URL:** <https://discourse.julialang.org/t/how-to-call-cuda-functions-that-are-not-implemented-in-cuda-jl/47219>\
**Category:** GPU\
**Created:** [September 24, 2020, 3:13pm UTC](https://discourse.julialang.org/t/how-to-call-cuda-functions-that-are-not-implemented-in-cuda-jl/47219 "2020-09-24T15:13:41Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![cuchxq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cuchxq/32/3830_2.png) [@cuchxq](https://discourse.julialang.org/u/cuchxq)\
**Post date:** [September 24, 2020, 3:13pm UTC](https://discourse.julialang.org/t/how-to-call-cuda-functions-that-are-not-implemented-in-cuda-jl/47219/1 "2020-09-24T15:13:41Z")

</div>

If there is a function in CUDA, but have not been implemented in CUDA.jl, is it possible to call that function using ccall?

I am trying it with a call to dgemm ( gemm is available through CUDA.jl, but I want to try a function that’s been implemented to validate the result), using the following script, the return value is `CUBLAS_STATUS_SUCCESS`, however, I don’t have a way to get the result, and the output indicates that the function didn’t actually succeed.

```julia
using LLVM
n = 10000

x = rand(n,n)
y = zeros(n,n)
alpha = rand()

handle = CUBLAS.cublasCreate_v2()

# CUBLAS.handle()
LLVM.@runtime_ccall((:cublasDgemm, CUDA.libcublas()), CUBLAS.cublasStatus_t, (CUBLAS.cublasHandle_t, Char, Char,
                                            Cint, Cint, Cint, 
                                            Cdouble, Ptr{Cdouble}, Ptr{Cdouble}, Cdouble, Ptr{Cdouble}, Cint),
                                            handle, 'N','N', n,n,n, alpha, x,x, alpha, y, n)

```

Here is the output:

```julia
# ** On entry to DGEMM parameter number 1 had an illegal value
# CUBLAS_STATUS_SUCCESS::cublasStatus_t = 0x00000000

```

The code probably is not correct. Any help are appreciated.

Thanks.

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [September 24, 2020, 3:47pm UTC](https://discourse.julialang.org/t/how-to-call-cuda-functions-that-are-not-implemented-in-cuda-jl/47219/2 "2020-09-24T15:47:25Z")

</div>

Functions that aren’t wrapped in CUDA.jl are still available through their autogenerated wrappers. So you can just call `CUBLAS.cublasDgemm`. However, the actual name is `cublasDgemm_v2`. ~~And that function _is_ being wrapped already, as `CUBLAS.gemm!` (see `wrappers.jl` in CUDA.jl/lib/cublas).~~

EDIT: didn’t read your question properly.

If the returned status is 0, everything was OK; where do you get the other error from?  
The ‘result’ here is the fact that this call modifies one of its inputs.

---

<div class="post-metadata">

**Author:** ![cuchxq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cuchxq/32/3830_2.png) [@cuchxq](https://discourse.julialang.org/u/cuchxq)\
**Post date:** [September 24, 2020, 4:50pm UTC](https://discourse.julialang.org/t/how-to-call-cuda-functions-that-are-not-implemented-in-cuda-jl/47219/3 "2020-09-24T16:50:14Z")

</div>

```julia

julia> LLVM.@runtime_ccall((:cublasDgemm, CUDA.libcublas()), CUBLAS.cublasStatus_t, (CUBLAS.cublasHandle_t, Char, Char,
                                                   Cint, Cint, Cint, 
                                                   Cdouble, Ptr{Cdouble}, Ptr{Cdouble}, Cdouble, Ptr{Cdouble}, Cint),
                                                   handle, 'N','N', n,n,n, alpha, x,x, alpha, y, n)
 ** On entry to DGEMM parameter number 1 had an illegal value
CUBLAS_STATUS_SUCCESS::cublasStatus_t = 0x00000000

```

The output just prints on screen after calling the function. I think it’s coming from CUDA, rather than Julia.

---

<div class="post-metadata">

**Author:** ![cuchxq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cuchxq/32/3830_2.png) [@cuchxq](https://discourse.julialang.org/u/cuchxq)\
**Post date:** [September 24, 2020, 4:52pm UTC](https://discourse.julialang.org/t/how-to-call-cuda-functions-that-are-not-implemented-in-cuda-jl/47219/4 "2020-09-24T16:52:59Z")

</div>

> [@maleadt](#):
>
> So you can just call `CUBLAS.cublasDgemm` .

I called this function:

```julia
julia> @time CUDA.@sync res = CUBLAS.cublasDgemm_v2(handle, 'T', 'N', n,n,n, alpha, x, n, x, n, alpha, y, n)
ERROR: MethodError: Cannot `convert` an object of type Char to an object of type CUDA.CUBLAS.cublasOperation_t
Closest candidates are:
  convert(::Type{T}, ::T) where T at essentials.jl:171
  CUDA.CUBLAS.cublasOperation_t(::Any) where T<:CEnum.Cenum at /home/hu/.julia/packages/CEnum/daZUb/src/operators.jl:19

```

So how to convert `Char` to `CUDA.CUBLAS.cublasOperation_t` ? Thank you much for your reply!

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [September 24, 2020, 5:11pm UTC](https://discourse.julialang.org/t/how-to-call-cuda-functions-that-are-not-implemented-in-cuda-jl/47219/5 "2020-09-24T17:11:28Z")

</div>

Look at how the existing wrappers do that. On CUDA master that convert method is available, for the version you’re using it’ll be explicit still.

---

<div class="post-metadata">

**Author:** ![cuchxq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cuchxq/32/3830_2.png) [@cuchxq](https://discourse.julialang.org/u/cuchxq)\
**Post date:** [September 24, 2020, 5:50pm UTC](https://discourse.julialang.org/t/how-to-call-cuda-functions-that-are-not-implemented-in-cuda-jl/47219/6 "2020-09-24T17:50:51Z")

</div>

The [util](https://github.com/JuliaGPU/CUDA.jl/blob/bd5a2a8800e91eb6a7df89eb5dd4bb8fc503541d/lib/cublas/util.jl) section shows that convert function is overloade, But it shows the same error:

```julia
julia> CUDA.convert(CUBLAS.cublasOperation_t, 'T')
ERROR: MethodError: Cannot `convert` an object of type Char to an object of type CUDA.CUBLAS.cublasOperation_t
Closest candidates are:
  convert(::Type{T}, ::T) where T at essentials.jl:171
  CUDA.CUBLAS.cublasOperation_t(::Any) where T<:CEnum.Cenum at /home/xiaoqihu/.julia/packages/CEnum/daZUb/src/operators.jl:19
Stacktrace:
 [1] top-level scope at REPL[7]:100:

```

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [September 24, 2020, 6:12pm UTC](https://discourse.julialang.org/t/how-to-call-cuda-functions-that-are-not-implemented-in-cuda-jl/47219/7 "2020-09-24T18:12:35Z")

</div>

And that works as expected:

```julia
julia> convert(CUBLAS.cublasOperation_t, 'T')
CUBLAS_OP_T::cublasOperation_t = 0x00000001

```

As I said, you need to check for the version of CUDA.jl you’re using. It used to be with a custom function, `cublasop`, and has been converted to more “Julian” interfaces nowadays. Just look at how `CUBLAS.gemm!` is implemented.

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [September 24, 2020, 6:20pm UTC](https://discourse.julialang.org/t/how-to-call-cuda-functions-that-are-not-implemented-in-cuda-jl/47219/8 "2020-09-24T18:20:59Z")

</div>

e.g. [https://github.com/JuliaGPU/CUDA.jl/blob/1c399bf2ed10624f3961d073f07d55b1a30184ef/lib/cublas/wrappers.jl#L818](https://github.com/JuliaGPU/CUDA.jl/blob/1c399bf2ed10624f3961d073f07d55b1a30184ef/lib/cublas/wrappers.jl#L818) on 1.1

---

<div class="post-metadata">

**Author:** ![cuchxq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cuchxq/32/3830_2.png) [@cuchxq](https://discourse.julialang.org/u/cuchxq)\
**Post date:** [September 25, 2020, 1:20am UTC](https://discourse.julialang.org/t/how-to-call-cuda-functions-that-are-not-implemented-in-cuda-jl/47219/9 "2020-09-25T01:20:26Z")

</div>

Thank you very much @maleadt! yes, the gemm! shows how to do it.