# CUDAnative test fails

**URL:** <https://discourse.julialang.org/t/cudanative-test-fails/38085>\
**Category:** GPU\
**Created:** [April 23, 2020, 1:36pm UTC](https://discourse.julialang.org/t/cudanative-test-fails/38085 "2020-04-23T13:36:01Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![eaubanel](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/eaubanel/32/13327_2.png) [@eaubanel](https://discourse.julialang.org/u/eaubanel)\
**Post date:** [April 23, 2020, 1:36pm UTC](https://discourse.julialang.org/t/cudanative-test-fails/38085/1 "2020-04-23T13:36:01Z")

</div>

I’m experienced with C/CUDA, but trying CUDA with Julia for the first time, on a compute node of a research cluster running CentOS Linux release 7.6.1810 and eight Tesla V100-SXM2 GPUs:

> **nvidia-smi**
>
> ±----------------------------------------------------------------------------+  
> | NVIDIA-SMI 440.33.01 Driver Version: 440.33.01 CUDA Version: 10.2 |  
> |-------------------------------±---------------------±---------------------+  
> | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |  
> | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |  
> |===============================+======================+======================|  
> | 0 Tesla V100-SXM2… Off | 00000000:1A:00.0 Off | 0 |  
> | N/A 27C P0 43W / 300W | 0MiB / 32510MiB | 0% Default |  
> ±------------------------------±---------------------±---------------------+  
> | 1 Tesla V100-SXM2… Off | 00000000:1B:00.0 Off | 0 |  
> | N/A 32C P0 41W / 300W | 0MiB / 32510MiB | 0% Default |  
> ±------------------------------±---------------------±---------------------+  
> | 2 Tesla V100-SXM2… Off | 00000000:3D:00.0 Off | 0 |  
> | N/A 31C P0 40W / 300W | 0MiB / 32510MiB | 0% Default |  
> ±------------------------------±---------------------±---------------------+  
> | 3 Tesla V100-SXM2… Off | 00000000:3E:00.0 Off | 0 |  
> | N/A 28C P0 42W / 300W | 0MiB / 32510MiB | 0% Default |  
> ±------------------------------±---------------------±---------------------+  
> | 4 Tesla V100-SXM2… Off | 00000000:88:00.0 Off | 0 |  
> | N/A 29C P0 41W / 300W | 0MiB / 32510MiB | 0% Default |  
> ±------------------------------±---------------------±---------------------+  
> | 5 Tesla V100-SXM2… Off | 00000000:89:00.0 Off | 0 |  
> | N/A 30C P0 43W / 300W | 0MiB / 32510MiB | 0% Default |  
> ±------------------------------±---------------------±---------------------+  
> | 6 Tesla V100-SXM2… Off | 00000000:B2:00.0 Off | 0 |  
> | N/A 31C P0 43W / 300W | 0MiB / 32510MiB | 0% Default |  
> ±------------------------------±---------------------±---------------------+  
> | 7 Tesla V100-SXM2… Off | 00000000:B3:00.0 Off | 0 |  
> | N/A 28C P0 43W / 300W | 0MiB / 32510MiB | 0% Default |  
> ±------------------------------±---------------------±---------------------+
> 
> ±----------------------------------------------------------------------------+  
> | Processes: GPU Memory |  
> | GPU PID Type Process name Usage |  
> |=============================================================================|  
> | No running processes found |  
> ±----------------------------------------------------------------------------+

I installed the CUDAdrv, CUDAnative, CuArrays packages successfully, and ran the Pkg.test on CUDAdrv and CuArrays successfully. However, the CUDAnative test wasn’t completely successfull:

```julia
julia> Pkg.test("CUDAnative")
    Testing CUDAnative
Status `/tmp/jl_0jNZ6d/Manifest.toml`
  [79e6a3ab] Adapt v1.0.1
  [b99e7846] BinaryProvider v0.5.8
  [fa961155] CEnum v0.2.0
  [3895d2a7] CUDAapi v4.0.0
  [c5f51814] CUDAdrv v6.2.2
  [be33ccc6] CUDAnative v3.0.4
  [da1fd8a2] CodeTracking v0.5.9
  [f68482b8] Cthulhu v1.0.2
  [864edb3b] DataStructures v0.17.13
  [e2ba6199] ExprTools v0.1.1
  [929cbde3] LLVM v1.3.4
  [bac558e1] OrderedCollections v1.1.0
  [a759f4b9] TimerOutputs v0.5.4
  [2a0f44e3] Base64 
  [ade2ca70] Dates 
  [8ba89e20] Distributed 
  [b77e0a4c] InteractiveUtils 
  [76f85450] LibGit2 
  [8f399da3] Libdl 
  [37e2e46d] LinearAlgebra 
  [56ddb016] Logging 
  [d6f4376e] Markdown 
  [44cfe95a] Pkg 
  [de0858da] Printf 
  [3fa0cd96] REPL 
  [9a3f8284] Random 
  [ea8e919c] SHA 
  [9e88b42a] Serialization 
  [6462fe0b] Sockets 
  [8dfed614] Test 
  [cf7118a7] UUIDs 
  [4ec0a83e] Unicode 
Downloading artifact: CUDA10.2

Downloading artifact: CUDA10.1

Downloading artifact: CUDA10.0

Downloading artifact: CUDA9.2

Downloading artifact: CUDA9.0

[ Info: Testing using device Tesla V100-SXM2-32GB (compute capability 7.0.0, 31.439 GiB available memory) on CUDA driver 10.2.0 and toolkit 10.0.130
stack traces at different debug levels: Test Failed at /home/eaubanel/.julia/packages/CUDAnative/ierw8/test/device/execution.jl:576
  Expression: isempty(out)
   Evaluated: isempty("Downloading artifact: CUDA10.2\n\e[?25l\e[1A\e[2K\e[?25hDownloading artifact: CUDA10.1\n\e[?25l\e[1A\e[2K\e[?25hDownloading artifact: CUDA10.0\n\e[?25l\e[1A\e[2K\e[?25hDownloading artifact: CUDA9.2\n\e[?25l\e[1A\e[2K\e[?25hDownloading artifact: CUDA9.0\n\e[?25l\e[1A\e[2K\e[?25h")
Stacktrace:
 [1] macro expansion at /home/eaubanel/.julia/packages/CUDAnative/ierw8/test/device/execution.jl:576 [inlined]
 [2] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Test/src/Test.jl:1113 [inlined]
 [3] macro expansion at /home/eaubanel/.julia/packages/CUDAnative/ierw8/test/device/execution.jl:560 [inlined]
 [4] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Test/src/Test.jl:1113 [inlined]
 [5] macro expansion at /home/eaubanel/.julia/packages/CUDAnative/ierw8/test/device/execution.jl:558 [inlined]
 [6] macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Test/src/Test.jl:1113 [inlined]
 [7] top-level scope at /home/eaubanel/.julia/packages/CUDAnative/ierw8/test/device/execution.jl:5
Test Summary: | Pass Fail Total
CUDAnative | 752 1 753
  base interface | No tests
  pointer | 20 20
  code generation | 84 84
  code generation (relying on a device) | 8 8
  execution | 89 1 90
    @cuda | 14 14
    argument passing | 30 30
    exceptions | 16 1 17
      stack traces at different debug levels | 11 1 12
      #329 | 5 5
    shmem divergence bug | 7 7
    dynamic parallelism | 11 11
    cooperative groups | 1 1
    threading | 10 10
  pointer | 37 37
  device arrays | 20 20
  CUDA functionality | 264 264
  WMMA | 206 206
  NVTX | No tests
  examples | 6 6
ERROR: LoadError: Some tests did not pass: 752 passed, 1 failed, 0 errored, 0 broken.
in expression starting at /home/eaubanel/.julia/packages/CUDAnative/ierw8/test/runtests.jl:12
ERROR: Package CUDAnative errored during testing
Stacktrace:
 [1] pkgerror(::String, ::Vararg{String,N} where N) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Pkg/src/Types.jl:53
 [2] test(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}; coverage::Bool, julia_args::Cmd, test_args::Cmd, test_fn::Nothing) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Pkg/src/Operations.jl:1503
 [3] test(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}; coverage::Bool, test_fn::Nothing, julia_args::Cmd, test_args::Cmd, kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:316
 [4] test(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:303
 [5] #test#68 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:297 [inlined]
 [6] test at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:297 [inlined]
 [7] #test#67 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:296 [inlined]
 [8] test at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:296 [inlined]
 [9] test(::String; kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:295
 [10] test(::String) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Pkg/src/API.jl:295
 [11] top-level scope at REPL[4]:1

```

> **versioninfo()**
>
> Julia Version 1.4.1  
> Commit 381693d3df\* (2020-04-14 17:20 UTC)  
> Platform Info:  
> OS: Linux (x86\_64-pc-linux-gnu)  
> CPU: Intel(R) Xeon(R) Platinum 8168 CPU @ 2.70GHz  
> WORD\_SIZE: 64  
> LIBM: libopenlibm  
> LLVM: libLLVM-8.0.1 (ORCJIT, skylake)

I can compile and run C/CUDA programs just fine on this node. Can anyone help?

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [April 23, 2020, 1:43pm UTC](https://discourse.julialang.org/t/cudanative-test-fails/38085/2 "2020-04-23T13:43:02Z")

</div>

The package works, it just started downloading when doing an output check and stumbled on the message.

That said, it should never have attempted to download all artifacts, since they are marked lazy. Unless each of those downloads failed, could it be that you can’t download things on those compute nodes? If so it’s better to run with `JULIA_CUDA_USE_BINARYBUILDER=false`, or make sure you download the artifacts on the login node.

---

<div class="post-metadata">

**Author:** ![eaubanel](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/eaubanel/32/13327_2.png) [@eaubanel](https://discourse.julialang.org/u/eaubanel)\
**Post date:** [April 23, 2020, 1:59pm UTC](https://discourse.julialang.org/t/cudanative-test-fails/38085/3 "2020-04-23T13:59:33Z")

</div>

Thanks, that was the problem. I used `JULIA_CUDA_USE_BINARYBUILDER=false` and the test passed.