Binary CUDA dependency

Specific Domains GPU
1

I’m in the process of building a Julia package which uses some custom CUDA kernels that I wrote a few years ago.

I want to provide these CUDA binaries with the package, and have them compiled using BinDeps.jl. I’m following a recipe that worked for some of my C++, but somehow BinDeps does not seem to be happy with my CUDA code.

Here is my build.jl file:

using BinDeps

@BinDeps.setup

deps = [
  cudautils   = library_dependency("cudautils")
]

prefix=joinpath(BinDeps.depsdir(cudautils))
linscan_aqdbuilddir = joinpath(BinDeps.depsdir(cudautils),"builds")

# === CUDA code ===
provides(BuildProcess,
    (@build_steps begin
        CreateDirectory(linscan_aqdbuilddir)
        @build_steps begin
            ChangeDirectory(linscan_aqdbuilddir)
            FileRule(joinpath(prefix,"builds","cudautils.ptx"),@build_steps begin
                `nvcc -ptx ../src/cudautils.cu -o cudautils.ptx -arch=compute_35`
            end)
        end
    end),cudautils, os = :Linux, installed_libpath=joinpath(prefix,"builds"))

@BinDeps.install Dict([(:cudautils, :cudautils)])

However, when I build I always get this message:

LoadError: Provider BinDeps.BuildProcess failed to satisfy dependency cudautils

It is not clear to me why this process works for C++ code that produces .so files, but not for CUDA code that produces .ptx files. Any help is appreciated.

2

I think BinDeps assumes your library ends with Libdl.dlext (which is probably .so). But it looks like you can add an alias with the .ptx extension:

library_dependency("cudautils", aliases=["cudautils.ptx"])

(assuming that the .ptx is still just a shared library. I don’t actually know anything about building CUDA libraries).

3

PTX files are not binaries (they’re assembly); nvcc can produce binaries if you don’t pass the ptx flag. How are you planning to link in/make use of the ptx files?

4

Correct, they are some sort of cuda mid-level representation.

I am linking to the .ptx files using CUDAdrv.jl’s CuModuleFile (e.g. something like this example).

using CUDAdrv
using Base.Test

using Compat

dev = CuDevice(0)
ctx = CuContext(dev)

md = CuModuleFile(joinpath(@__DIR__, "vadd.ptx"))
vadd = CuFunction(md, "kernel_vadd")

dims = (3,4)
a = round.(rand(Float32, dims) * 100)
b = round.(rand(Float32, dims) * 100)

d_a = CuArray(a)
d_b = CuArray(b)
d_c = similar(d_a)

len = prod(dims)
cudacall(vadd, len, 1, Tuple{Ptr{Cfloat},Ptr{Cfloat},Ptr{Cfloat}}, d_a, d_b, d_c)
c = Array(d_c)
@test a+b ≈ c

destroy!(ctx)
5

Oh, in that case I’m definitely wrong, so please ignore the noise :slight_smile:

1 Like
6

I think that means you might be right, since the problem appears to be simply that BinDeps can’t tell the requirement is satisfied because the extension is unexpected?

7

That might be correct. I tried changing the extension to .so to see if I could “fool” BinDeps, but it didn’t work.

But you are right, at the heart of my confusion is the fact that I cannot seem to find what criteria are necessary for BinDeps to mark a dependency as “satisfied”, and the documentation and error outputs are not super helpful in that sense.

8

Yeah, I agree that the documentation is sparse. The library is tested here: https://github.com/JuliaLang/BinDeps.jl/blob/master/src/dependencies.jl#L652 so it needs to be openable with dlopen()

9

Some alternatives: you could use libnvrtc to compile CUDA C to PTX, but we don’t have that wrapped yet.
Or commit compiled PTX code, just make sure to target an old-enough PTX ISA.

Fixing Bindeps seems preferable though. Do note that nvcc isn’t necessarily on the PATH though, that’s (among other things) why we have CUDAapi.jl.