Try using CSR matrix and csrlsvqr!. Here’s an example:
using SparseArrays
using LinearAlgebra
using CUDA
n = 100
a = sprand(n,n,0.2) + sparse(I, n,n)
A = CUSPARSE.CuSparseMatrixCSR(a)
b = CUDA.rand(Float64, n)
x = CuVector{Float64}(undef,n)
tol = 1e-8
CUSOLVER.csrlsvqr!(A, b, x, tol, one(Cint),'O')
See https://github.com/JuliaGPU/CUDA.jl/blob/master/test/cusolver/sparse.jl