Binary CUDA dependency

I’m in the process of building a Julia package which uses some custom CUDA kernels that I wrote a few years ago.

I want to provide these CUDA binaries with the package, and have them compiled using BinDeps.jl. I’m following a recipe that worked for some of my C++, but somehow BinDeps does not seem to be happy with my CUDA code.

Here is my build.jl file:

using BinDeps


deps = [
  cudautils   = library_dependency("cudautils")

linscan_aqdbuilddir = joinpath(BinDeps.depsdir(cudautils),"builds")

# === CUDA code ===
    (@build_steps begin
        @build_steps begin
            FileRule(joinpath(prefix,"builds","cudautils.ptx"),@build_steps begin
                `nvcc -ptx ../src/ -o cudautils.ptx -arch=compute_35`
    end),cudautils, os = :Linux, installed_libpath=joinpath(prefix,"builds"))

@BinDeps.install Dict([(:cudautils, :cudautils)])

However, when I build I always get this message:

LoadError: Provider BinDeps.BuildProcess failed to satisfy dependency cudautils

It is not clear to me why this process works for C++ code that produces .so files, but not for CUDA code that produces .ptx files. Any help is appreciated.

I think BinDeps assumes your library ends with Libdl.dlext (which is probably .so). But it looks like you can add an alias with the .ptx extension:

library_dependency("cudautils", aliases=["cudautils.ptx"])

(assuming that the .ptx is still just a shared library. I don’t actually know anything about building CUDA libraries).

PTX files are not binaries (they’re assembly); nvcc can produce binaries if you don’t pass the ptx flag. How are you planning to link in/make use of the ptx files?

Correct, they are some sort of cuda mid-level representation.

I am linking to the .ptx files using CUDAdrv.jl’s CuModuleFile (e.g. something like this example).

using CUDAdrv
using Base.Test

using Compat

dev = CuDevice(0)
ctx = CuContext(dev)

md = CuModuleFile(joinpath(@__DIR__, "vadd.ptx"))
vadd = CuFunction(md, "kernel_vadd")

dims = (3,4)
a = round.(rand(Float32, dims) * 100)
b = round.(rand(Float32, dims) * 100)

d_a = CuArray(a)
d_b = CuArray(b)
d_c = similar(d_a)

len = prod(dims)
cudacall(vadd, len, 1, Tuple{Ptr{Cfloat},Ptr{Cfloat},Ptr{Cfloat}}, d_a, d_b, d_c)
c = Array(d_c)
@test a+b ≈ c


Oh, in that case I’m definitely wrong, so please ignore the noise :slight_smile:

1 Like

I think that means you might be right, since the problem appears to be simply that BinDeps can’t tell the requirement is satisfied because the extension is unexpected?

That might be correct. I tried changing the extension to .so to see if I could “fool” BinDeps, but it didn’t work.

But you are right, at the heart of my confusion is the fact that I cannot seem to find what criteria are necessary for BinDeps to mark a dependency as “satisfied”, and the documentation and error outputs are not super helpful in that sense.

Yeah, I agree that the documentation is sparse. The library is tested here: so it needs to be openable with dlopen()

Some alternatives: you could use libnvrtc to compile CUDA C to PTX, but we don’t have that wrapped yet.
Or commit compiled PTX code, just make sure to target an old-enough PTX ISA.

Fixing Bindeps seems preferable though. Do note that nvcc isn’t necessarily on the PATH though, that’s (among other things) why we have CUDAapi.jl.