Error when setting up Flux and CuArrays

John_G · September 23, 2019, 3:27pm

I get the following error whenever I run using Flux or using CuArrays, but I haven’t gotten the error when running any other Julia packages so far. Honestly I have no idea what it means or what to do about it, and any help would be appreciated.

The error is as follows:

(Error Details: Click to expand)

 Info: Recompiling stale cache file /home/gilbert/.julia/compiled/v1.0/Flux/QdkVy.ji for Flux [587475ba-b771-5e3f-ad9e-33799f191a9c]
└ @ Base loading.jl:1190
Internal error: encountered unexpected error in runtime:
BoundsError(a=Array{Core.Compiler.BasicBlock, (32,)}[
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=1, last=7), preds=Array{Int64, (1,)}[32], succs=Array{Int64, (1,)}[2]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=8, last=14), preds=Array{Int64, (1,)}[1], succs=Array{Int64, (2,)}[5, 3]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=15, last=20), preds=Array{Int64, (1,)}[2], succs=Array{Int64, (1,)}[4]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=21, last=21), preds=Array{Int64, (1,)}[3], succs=Array{Int64, (1,)}[7]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=22, last=28), preds=Array{Int64, (1,)}[2], succs=Array{Int64, (1,)}[6]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=29, last=29), preds=Array{Int64, (1,)}[5], succs=Array{Int64, (1,)}[7]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=30, last=47), preds=Array{Int64, (2,)}[4, 6], succs=Array{Int64, (2,)}[9, 8]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=48, last=49), preds=Array{Int64, (1,)}[7], succs=Array{Int64, (0,)}[]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=50, last=50), preds=Array{Int64, (1,)}[7], succs=Array{Int64, (1,)}[10]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=51, last=52), preds=Array{Int64, (1,)}[9], succs=Array{Int64, (1,)}[11]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=53, last=53), preds=Array{Int64, (1,)}[10], succs=Array{Int64, (1,)}[12]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=54, last=54), preds=Array{Int64, (1,)}[11], succs=Array{Int64, (1,)}[13]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=55, last=56), preds=Array{Int64, (1,)}[12], succs=Array{Int64, (1,)}[14]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=57, last=65), preds=Array{Int64, (1,)}[13], succs=Array{Int64, (1,)}[15]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=66, last=71), preds=Array{Int64, (1,)}[14], succs=Array{Int64, (2,)}[17, 16]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=72, last=74), preds=Array{Int64, (1,)}[15], succs=Array{Int64, (0,)}[]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=75, last=82), preds=Array{Int64, (1,)}[15], succs=Array{Int64, (2,)}[19, 18]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=83, last=84), preds=Array{Int64, (1,)}[17], succs=Array{Int64, (0,)}[]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=85, last=85), preds=Array{Int64, (1,)}[17], succs=Array{Int64, (1,)}[20]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=86, last=87), preds=Array{Int64, (1,)}[19], succs=Array{Int64, (1,)}[21]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=88, last=88), preds=Array{Int64, (1,)}[20], succs=Array{Int64, (1,)}[22]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=89, last=89), preds=Array{Int64, (1,)}[21], succs=Array{Int64, (1,)}[23]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=90, last=90), preds=Array{Int64, (1,)}[22], succs=Array{Int64, (1,)}[24]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=91, last=92), preds=Array{Int64, (1,)}[23], succs=Array{Int64, (1,)}[25]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=93, last=93), preds=Array{Int64, (1,)}[24], succs=Array{Int64, (1,)}[26]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=94, last=94), preds=Array{Int64, (1,)}[25], succs=Array{Int64, (1,)}[27]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=95, last=95), preds=Array{Int64, (1,)}[26], succs=Array{Int64, (2,)}[29, 28]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=96, last=98), preds=Array{Int64, (1,)}[27], succs=Array{Int64, (0,)}[]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=99, last=102), preds=Array{Int64, (1,)}[27], succs=Array{Int64, (2,)}[31, 30]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=103, last=103), preds=Array{Int64, (1,)}[29], succs=Array{Int64, (1,)}[32]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=104, last=104), preds=Array{Int64, (1,)}[29], succs=Array{Int64, (1,)}[32]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=105, last=107), preds=Array{Int64, (2,)}[30, 31], succs=Array{Int64, (1,)}[1])], i=(0,))
rec_backtrace at /usr/local/src/conda/julia-1.0.3/src/stackwalk.c:94
record_backtrace at /usr/local/src/conda/julia-1.0.3/src/task.c:246
jl_throw at /usr/local/src/conda/julia-1.0.3/src/task.c:577
jl_bounds_error_ints at /usr/local/src/conda/julia-1.0.3/src/rtutils.c:187
getindex at ./array.jl:731
jfptr_getindex_1594 at /home/gilbert/anaconda3/envs/econda/lib/julia/sys.so (unknown line)
jl_apply_generic at /usr/local/src/conda/julia-1.0.3/src/gf.c:2184
replace_code_newstyle! at ./compiler/ssair/legacy.jl:80
optimize at ./compiler/optimize.jl:212
typeinf at ./compiler/typeinfer.jl:35
typeinf_ext at ./compiler/typeinfer.jl:574
typeinf_ext at ./compiler/typeinfer.jl:611
jfptr_typeinf_ext_1 at /home/gilbert/anaconda3/envs/econda/lib/julia/sys.so (unknown line)
jl_apply_generic at /usr/local/src/conda/julia-1.0.3/src/gf.c:2184
jl_apply at /usr/local/src/conda/julia-1.0.3/src/julia.h:1537 [inlined]
jl_apply_with_saved_exception_state at /usr/local/src/conda/julia-1.0.3/src/rtutils.c:257
jl_type_infer at /usr/local/src/conda/julia-1.0.3/src/gf.c:275
jl_compile_method_internal at /usr/local/src/conda/julia-1.0.3/src/gf.c:1786 [inlined]
jl_fptr_trampoline at /usr/local/src/conda/julia-1.0.3/src/gf.c:1830
jl_apply_generic at /usr/local/src/conda/julia-1.0.3/src/gf.c:2184
jl_apply at /usr/local/src/conda/julia-1.0.3/src/julia.h:1537 [inlined]
start_task at /usr/local/src/conda/julia-1.0.3/src/task.c:268
unknown function (ip: 0xffffffffffffffff)
Internal error: encountered unexpected error in runtime:
BoundsError(a=Array{Core.Compiler.BasicBlock, (32,)}[
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=1, last=7), preds=Array{Int64, (1,)}[32], succs=Array{Int64, (1,)}[2]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=8, last=14), preds=Array{Int64, (1,)}[1], succs=Array{Int64, (2,)}[5, 3]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=15, last=20), preds=Array{Int64, (1,)}[2], succs=Array{Int64, (1,)}[4]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=21, last=21), preds=Array{Int64, (1,)}[3], succs=Array{Int64, (1,)}[7]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=22, last=28), preds=Array{Int64, (1,)}[2], succs=Array{Int64, (1,)}[6]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=29, last=29), preds=Array{Int64, (1,)}[5], succs=Array{Int64, (1,)}[7]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=30, last=47), preds=Array{Int64, (2,)}[4, 6], succs=Array{Int64, (2,)}[9, 8]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=48, last=49), preds=Array{Int64, (1,)}[7], succs=Array{Int64, (0,)}[]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=50, last=50), preds=Array{Int64, (1,)}[7], succs=Array{Int64, (1,)}[10]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=51, last=52), preds=Array{Int64, (1,)}[9], succs=Array{Int64, (1,)}[11]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=53, last=53), preds=Array{Int64, (1,)}[10], succs=Array{Int64, (1,)}[12]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=54, last=54), preds=Array{Int64, (1,)}[11], succs=Array{Int64, (1,)}[13]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=55, last=56), preds=Array{Int64, (1,)}[12], succs=Array{Int64, (1,)}[14]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=57, last=65), preds=Array{Int64, (1,)}[13], succs=Array{Int64, (1,)}[15]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=66, last=71), preds=Array{Int64, (1,)}[14], succs=Array{Int64, (2,)}[17, 16]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=72, last=74), preds=Array{Int64, (1,)}[15], succs=Array{Int64, (0,)}[]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=75, last=82), preds=Array{Int64, (1,)}[15], succs=Array{Int64, (2,)}[19, 18]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=83, last=84), preds=Array{Int64, (1,)}[17], succs=Array{Int64, (0,)}[]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=85, last=85), preds=Array{Int64, (1,)}[17], succs=Array{Int64, (1,)}[20]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=86, last=87), preds=Array{Int64, (1,)}[19], succs=Array{Int64, (1,)}[21]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=88, last=88), preds=Array{Int64, (1,)}[20], succs=Array{Int64, (1,)}[22]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=89, last=89), preds=Array{Int64, (1,)}[21], succs=Array{Int64, (1,)}[23]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=90, last=90), preds=Array{Int64, (1,)}[22], succs=Array{Int64, (1,)}[24]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=91, last=92), preds=Array{Int64, (1,)}[23], succs=Array{Int64, (1,)}[25]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=93, last=93), preds=Array{Int64, (1,)}[24], succs=Array{Int64, (1,)}[26]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=94, last=94), preds=Array{Int64, (1,)}[25], succs=Array{Int64, (1,)}[27]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=95, last=95), preds=Array{Int64, (1,)}[26], succs=Array{Int64, (2,)}[29, 28]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=96, last=98), preds=Array{Int64, (1,)}[27], succs=Array{Int64, (0,)}[]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=99, last=102), preds=Array{Int64, (1,)}[27], succs=Array{Int64, (2,)}[31, 30]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=103, last=103), preds=Array{Int64, (1,)}[29], succs=Array{Int64, (1,)}[32]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=104, last=104), preds=Array{Int64, (1,)}[29], succs=Array{Int64, (1,)}[32]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=105, last=107), preds=Array{Int64, (2,)}[30, 31], succs=Array{Int64, (1,)}[1])], i=(0,))
rec_backtrace at /usr/local/src/conda/julia-1.0.3/src/stackwalk.c:94
record_backtrace at /usr/local/src/conda/julia-1.0.3/src/task.c:246
jl_throw at /usr/local/src/conda/julia-1.0.3/src/task.c:577
jl_bounds_error_ints at /usr/local/src/conda/julia-1.0.3/src/rtutils.c:187
getindex at ./array.jl:731
jfptr_getindex_1594 at /home/gilbert/anaconda3/envs/econda/lib/julia/sys.so (unknown line)
jl_apply_generic at /usr/local/src/conda/julia-1.0.3/src/gf.c:2184
replace_code_newstyle! at ./compiler/ssair/legacy.jl:80
optimize at ./compiler/optimize.jl:212
typeinf at ./compiler/typeinfer.jl:35
typeinf_ext at ./compiler/typeinfer.jl:574
typeinf_ext at ./compiler/typeinfer.jl:611
jfptr_typeinf_ext_1 at /home/gilbert/anaconda3/envs/econda/lib/julia/sys.so (unknown line)
jl_apply_generic at /usr/local/src/conda/julia-1.0.3/src/gf.c:2184
jl_apply at /usr/local/src/conda/julia-1.0.3/src/julia.h:1537 [inlined]
jl_apply_with_saved_exception_state at /usr/local/src/conda/julia-1.0.3/src/rtutils.c:257
jl_type_infer at /usr/local/src/conda/julia-1.0.3/src/gf.c:275
jl_compile_method_internal at /usr/local/src/conda/julia-1.0.3/src/gf.c:1786 [inlined]
jl_fptr_trampoline at /usr/local/src/conda/julia-1.0.3/src/gf.c:1830
jl_apply_generic at /usr/local/src/conda/julia-1.0.3/src/gf.c:2184
jl_apply at /usr/local/src/conda/julia-1.0.3/src/julia.h:1537 [inlined]
start_task at /usr/local/src/conda/julia-1.0.3/src/task.c:268
unknown function (ip: 0xffffffffffffffff)

John_G · September 23, 2019, 3:29pm

These are part of the results of running using Pkg; Pkg.test("Flux") and the results of Pkg.test("CuArrays"):

(Flux Test Results: Click to expand)

[ Info: Testing Flux/CUDNN
batch_size = 1: Error During Test at /home/gilbert/.julia/packages/Flux/dkJUV/test/cuda/curnn.jl:7
  Got exception outside of a @test
  CUDNNError(code 3, CUDNN_STATUS_BAD_PARAM)
  Stacktrace:
   [1] macro expansion at /home/gilbert/.julia/packages/CuArrays/wXQp8/src/dnn/error.jl:19 [inlined]
   [2] cudnnRNNBackwardData(::Flux.CUDA.RNNDesc{Float32}, ::Int64, ::Array{CuArrays.CUDNN.TensorDesc,1}, ::CuArray{Float32,1}, ::Array{CuArrays.CUDNN.TensorDesc,1}, ::CuArray{Float32,1}, ::CuArrays.CUDNN.TensorDesc, ::CuArray{Float32,1}, ::Ptr{Nothing}, ::CUDAdrv.CuPtr{Nothing}, ::CuArrays.CUDNN.FilterDesc, ::CuArray{Float32,1}, ::CuArrays.CUDNN.TensorDesc, ::CuArray{Float32,1}, ::Ptr{Nothing}, ::CUDAdrv.CuPtr{Nothing}, ::Array{CuArrays.CUDNN.TensorDesc,1}, ::CuArray{Float32,1}, ::CuArrays.CUDNN.TensorDesc, ::CuArray{Float32,1}, ::Ptr{Nothing}, ::CUDAdrv.CuPtr{Nothing}, ::CuArray{UInt8,1}, ::CuArray{UInt8,1}) at /home/gilbert/.julia/packages/Flux/dkJUV/src/cuda/curnn.jl:174
   [3] backwardData(::Flux.CUDA.RNNDesc{Float32}, ::CuArray{Float32,1}, ::CuArray{Float32,1}, ::CuArray{Float32,1}, ::Nothing, ::CuArray{Float32,1}, ::Nothing, ::CuArray{UInt8,1}) at /home/gilbert/.julia/packages/Flux/dkJUV/src/cuda/curnn.jl:191
   [4] backwardData(::Flux.CUDA.RNNDesc{Float32}, ::CuArray{Float32,1}, ::CuArray{Float32,1}, ::CuArray{Float32,1}, ::CuArray{Float32,1}, ::CuArray{UInt8,1}) at /home/gilbert/.julia/packages/Flux/dkJUV/src/cuda/curnn.jl:199
   [5] (::getfield(Flux.CUDA, Symbol("##8#9")){Flux.GRUCell{TrackedArray{…,CuArray{Float32,2}},TrackedArray{…,CuArray{Float32,1}}},TrackedArray{…,CuArray{Float32,1}},TrackedArray{…,CuArray{Float32,1}},CuArray{UInt8,1},Tuple{CuArray{Float32,1},CuArray{Float32,1}}})(::Tuple{CuArray{Float32,1},CuArray{Float32,1}}) at /home/gilbert/.julia/packages/Flux/dkJUV/src/cuda/curnn.jl:310
   [6] back_(::Tracker.Call{getfield(Flux.CUDA, Symbol("##8#9")){Flux.GRUCell{TrackedArray{…,CuArray{Float32,2}},TrackedArray{…,CuArray{Float32,1}}},TrackedArray{…,CuArray{Float32,1}},TrackedArray{…,CuArray{Float32,1}},CuArray{UInt8,1},Tuple{CuArray{Float32,1},CuArray{Float32,1}}},Tuple{Tracker.Tracked{CuArray{Float32,1}},Tracker.Tracked{CuArray{Float32,1}},Tracker.Tracked{CuArray{Float32,2}},Tracker.Tracked{CuArray{Float32,2}},Tracker.Tracked{CuArray{Float32,1}}}}, ::Tuple{CuArray{Float32,1},CuArray{Float32,1}}, ::Bool) at /home/gilbert/.julia/packages/Tracker/SAr25/src/back.jl:35
   [7] back(::Tracker.Tracked{Tuple{CuArray{Float32,1},CuArray{Float32,1}}}, ::Tuple{CuArray{Float32,1},Int64}, ::Bool) at/home/gilbert/.julia/packages/Tracker/SAr25/src/back.jl:58
   [8] (::getfield(Tracker, Symbol("##13#14")){Bool})(::Tracker.Tracked{Tuple{CuArray{Float32,1},CuArray{Float32,1}}}, ::Tuple{CuArray{Float32,1},Int64}) at /home/gilbert/.julia/packages/Tracker/SAr25/src/back.jl:38
   [9] foreach(::Function, ::Tuple{Tracker.Tracked{Tuple{CuArray{Float32,1},CuArray{Float32,1}}},Nothing}, ::Tuple{Tuple{CuArray{Float32,1},Int64},Nothing}) at ./abstractarray.jl:1836
   [10] back_(::Tracker.Call{getfield(Tracker, Symbol("##361#363")){Tracker.TrackedTuple{Tuple{CuArray{Float32,1},CuArray{Float32,1}}},Int64},Tuple{Tracker.Tracked{Tuple{CuArray{Float32,1},CuArray{Float32,1}}},Nothing}}, ::CuArray{Float32,1}, ::Bool) at /home/gilbert/.julia/packages/Tracker/SAr25/src/back.jl:38
   [11] back(::Tracker.Tracked{CuArray{Float32,1}}, ::CuArray{Float32,1}, ::Bool) at /home/gilbert/.julia/packages/Tracker/SAr25/src/back.jl:58
   [12] back!(::TrackedArray{…,CuArray{Float32,1}}, ::CuArray{Float32,1}) at /home/gilbert/.julia/packages/Tracker/SAr25/src/back.jl:77
   [13] macro expansion at /home/gilbert/.julia/packages/Flux/dkJUV/test/cuda/curnn.jl:23 [inlined]
   [14] macro expansion at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1156 [inlined]
   [15] macro expansion at /home/gilbert/.julia/packages/Flux/dkJUV/test/cuda/curnn.jl:7 [inlined]
   [16] macro expansion at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1156 [inlined]
   [17] macro expansion at /home/gilbert/.julia/packages/Flux/dkJUV/test/cuda/curnn.jl:4 [inlined]
   [18] macro expansion at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
   [19] top-level scope at /home/gilbert/.julia/packages/Flux/dkJUV/test/cuda/curnn.jl:4
   [20] include at ./boot.jl:317 [inlined]
   [21] include_relative(::Module, ::String) at ./loading.jl:1044
   [22] include(::Module, ::String) at ./sysimg.jl:29
   [23] include(::String) at ./client.jl:392
   [24] top-level scope at /home/gilbert/.julia/packages/Flux/dkJUV/test/cuda/cuda.jl:55
   [25] include at ./boot.jl:317 [inlined]
   [26] include_relative(::Module, ::String) at ./loading.jl:1044
   [27] include(::Module, ::String) at ./sysimg.jl:29
   [28] include(::String) at ./client.jl:392
   [29] macro expansion at /home/gilbert/.julia/packages/Flux/dkJUV/test/runtests.jl:30 [inlined]
   [30] macro expansion at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
   [31] top-level scope at /home/gilbert/.julia/packages/Flux/dkJUV/test/runtests.jl:11
   [32] include at ./boot.jl:317 [inlined]
   [33] include_relative(::Module, ::String) at ./loading.jl:1044
   [34] include(::Module, ::String) at ./sysimg.jl:29
   [35] include(::String) at ./client.jl:392
   [36] top-level scope at none:0
   [37] eval(::Module, ::Any) at ./boot.jl:319
   [38] exec_options(::Base.JLOptions) at ./client.jl:243
   [39] _start() at ./client.jl:425
batch_size = 5: Test Failed at /home/gilbert/.julia/packages/Flux/dkJUV/test/cuda/curnn.jl:26
  Expression: ((rnn.cell).Wi).grad ≈ collect(((curnn.cell).Wi).grad)
   Evaluated: Float32[0.0264221 0.0311623 … 0.0401529 0.0481648; -0.00580488 -0.00684245 … -0.000633113 -0.00512767; … ; -0.488995 -0.397307 … -0.543058 -0.593239; -1.59764 -1.72794 … -1.24722 -2.0625] ≈ Float32[0.0163969 0.0200817 … 0.0357488 0.036319; -0.00383122 -0.00466102 … 0.000233916 -0.00279561; … ; -0.655613 -0.581466 … -0.616253 -0.790114; -1.32217 -1.42347 … -1.1262 -1.737]
Stacktrace:
 [1] macro expansion at /home/gilbert/.julia/packages/Flux/dkJUV/test/cuda/curnn.jl:26 [inlined]
 [2] macro expansion at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1156 [inlined]
 [3] macro expansion at /home/gilbert/.julia/packages/Flux/dkJUV/test/cuda/curnn.jl:7 [inlined]
 [4] macro expansion at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1156 [inlined]
 [5] macro expansion at /home/gilbert/.julia/packages/Flux/dkJUV/test/cuda/curnn.jl:4 [inlined]
 [6] macro expansion at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
 [7] top-level scope at /home/gilbert/.julia/packages/Flux/dkJUV/test/cuda/curnn.jl:4
batch_size = 5: Test Failed at /home/gilbert/.julia/packages/Flux/dkJUV/test/cuda/curnn.jl:27
  Expression: ((rnn.cell).Wh).grad ≈ collect(((curnn.cell).Wh).grad)
   Evaluated: Float32[-0.0155654 0.00326925 … -0.0217709 -0.00794538; 0.00331486 -0.000961465 … 0.00208863 -0.000682507; …; 0.0600955 -0.0379895 … 0.0877803 0.056216; 0.13519 -0.049922 … 0.1269 0.0464798] ≈ Float32[-0.0102084 0.00063507 … -0.0177858 -0.00776742; 0.00226024 -0.000442876 … 0.00130408 -0.000717541; … ; 0.0800566 -0.047805 … 0.10263 0.0568791; 0.120198-0.04255 … 0.115747 0.0459818]
Stacktrace:
 [1] macro expansion at /home/gilbert/.julia/packages/Flux/dkJUV/test/cuda/curnn.jl:27 [inlined]
 [2] macro expansion at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1156 [inlined]
 [3] macro expansion at /home/gilbert/.julia/packages/Flux/dkJUV/test/cuda/curnn.jl:7 [inlined]
 [4] macro expansion at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1156 [inlined]
 [5] macro expansion at /home/gilbert/.julia/packages/Flux/dkJUV/test/cuda/curnn.jl:4 [inlined]
 [6] macro expansion at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
 [7] top-level scope at /home/gilbert/.julia/packages/Flux/dkJUV/test/cuda/curnn.jl:4
batch_size = 5: Test Failed at /home/gilbert/.julia/packages/Flux/dkJUV/test/cuda/curnn.jl:28
  Expression: ((rnn.cell).b).grad ≈ collect(((curnn.cell).b).grad)
   Evaluated: Float32[0.0431294, -0.00524064, 0.00328069, 0.0383973, -0.0277038, -0.190451, 0.378964, 0.113537, -1.10465, -0.109885, -0.380812, -0.806893, -0.591657, -0.840546, -2.16732] ≈ Float32[0.0309576, -0.0028444, 0.00784941, 0.0508012, -0.0251556, -0.116478, 0.196908, 0.156512, -1.30342, -0.109, -0.0338723, -0.537076, -0.840815, -1.04284, -1.83286]
Stacktrace:
 [1] macro expansion at /home/gilbert/.julia/packages/Flux/dkJUV/test/cuda/curnn.jl:28 [inlined]
 [2] macro expansion at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1156 [inlined]
 [3] macro expansion at /home/gilbert/.julia/packages/Flux/dkJUV/test/cuda/curnn.jl:7 [inlined]
 [4] macro expansion at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1156 [inlined]
 [5] macro expansion at /home/gilbert/.julia/packages/Flux/dkJUV/test/cuda/curnn.jl:4 [inlined]
 [6] macro expansion at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
 [7] top-level scope at /home/gilbert/.julia/packages/Flux/dkJUV/test/cuda/curnn.jl:4
batch_size = 5: Test Failed at /home/gilbert/.julia/packages/Flux/dkJUV/test/cuda/curnn.jl:29
  Expression: ((rnn.cell).h).grad ≈ collect(((curnn.cell).h).grad)
   Evaluated: Float32[-0.0128217, -0.706767, -0.358611, -1.41596, -0.547122] ≈ Float32[0.0667142, -0.127551, -0.454483, -1.79769, -0.414044]
Stacktrace:
 [1] macro expansion at /home/gilbert/.julia/packages/Flux/dkJUV/test/cuda/curnn.jl:29 [inlined]
 [2] macro expansion at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1156 [inlined]
 [3] macro expansion at /home/gilbert/.julia/packages/Flux/dkJUV/test/cuda/curnn.jl:7 [inlined]
 [4] macro expansion at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1156 [inlined]
 [5] macro expansion at /home/gilbert/.julia/packages/Flux/dkJUV/test/cuda/curnn.jl:4 [inlined]
 [6] macro expansion at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
 [7] top-level scope at /home/gilbert/.julia/packages/Flux/dkJUV/test/cuda/curnn.jl:4
Test Summary:                         | Pass  Fail  Error  Total
Flux                                  |  274     4      1    279
  Throttle                            |   11                  11
  Jacobian                            |    1                   1
  Initialization                      |   12                  12
  Params                              |    4                   4
  Basic Stacking                      |    1                   1
  Precision                           |    6                   6
  Stacking                            |    3                   3
  onecold                             |    4                   4
  onehotbatch indexing                |    2                   2
  Optimise                            |   11                  11
  Optimiser                           |    3                   3
  Training Loop                       |    2                   2
  ExpDecay                            |    3                   3
  basic                               |   27                  27
  Dropout                             |   10                  10
  BatchNorm                           |   14                  14
  InstanceNorm                        |   16                  16
  GroupNorm                           |   16                  16
  losses                              |   30                  30
  Pooling                             |    2                   2
  CNN                                 |    1                   1
  asymmetric padding                  |    7                   7
  Depthwise Conv                      |    3                   3
  ConvTranspose                       |    1                   1
  CrossCor                            |    4                   4
  Conv with non quadratic window #700 |    4                   4
  Tracker                             |    4                   4
  CuArrays                            |    8                   8
  onecold gpu                         |    2                   2
  CUDNN BatchNorm                     |   10                  10
  RNN                                 |   40     4      1     45
    R = Flux.RNN                      |   16                  16
    R = Flux.GRU                      |    6     4      1     11
      batch_size = 1                  |    2            1      3
      batch_size = 5                  |    4     4             8
    R = Flux.LSTM                     |   18                  18
ERROR: LoadError: Some tests did not pass: 274 passed, 4 failed, 1 errored, 0 broken.
in expression starting at /home/gilbert/.julia/packages/Flux/dkJUV/test/runtests.jl:9
ERROR: Package Flux errored during testing
Stacktrace:
 [1] pkgerror(::String, ::Vararg{String,N} where N) at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Pkg/src/Types.jl:120
 [2] #test#65(::Bool, ::Function, ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Pkg/src/Operations.jl:1357
 [3] #test at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:0 [inlined]
 [4] #test#42(::Bool, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:273
 [5] test at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:258 [inlined]
 [6] #test#41 at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:255 [inlined]
 [7] test at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:255 [inlined]
 [8] #test#40 at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:254 [inlined]
 [9] test at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:254 [inlined]
 [10] #test#39 at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:253 [inlined]
 [11] test(::String) at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:253
 [12] top-level scope at none:0

John_G · September 23, 2019, 3:30pm

(CuArrays Test Results: Click to expand)

Pkg.test("CuArrays")
   Testing CuArrays
 Resolving package versions...
    Status `/tmp/tmpXvVEZP/Manifest.toml`
  [621f4979] AbstractFFTs v0.4.1
  [79e6a3ab] Adapt v1.0.0
  [9e28174c] BinDeps v0.8.10
  [b99e7846] BinaryProvider v0.5.6
  [fa961155] CEnum v0.2.0
  [00ebfdb7] CSTParser v0.6.2
  [3895d2a7] CUDAapi v1.2.0
  [c5f51814] CUDAdrv v3.1.0
  [be33ccc6] CUDAnative v2.3.1
  [bbf7d656] CommonSubexpressions v0.2.0
  [34da2185] Compat v2.1.0
  [8f4d0f93] Conda v1.3.0
  [a8cc5b0e] Crayons v4.0.0
  [3a865a2d] CuArrays v1.2.1
  [864edb3b] DataStructures v0.17.0
  [163ba53b] DiffResults v0.0.4
  [b552c78f] DiffRules v0.0.10
  [7a1cc6ca] FFTW v0.3.0
  [1a297f60] FillArrays v0.6.4
  [f6369f11] ForwardDiff v0.10.3
  [0c68f7d7] GPUArrays v1.0.3
  [682c06a0] JSON v0.21.0
  [929cbde3] LLVM v1.3.0
  [1914dd2f] MacroTools v0.5.1
  [872c559c] NNlib v0.6.0
  [77ba4419] NaNMath v0.3.2
  [bac558e1] OrderedCollections v1.1.0
  [69de0a69] Parsers v0.3.7
  [189a3867] Reexport v0.2.0
  [ae029012] Requires v0.5.2
  [276daf66] SpecialFunctions v0.8.0
  [90137ffa] StaticArrays v0.11.0
  [a759f4b9] TimerOutputs v0.5.0
  [0796e94c] Tokenize v0.5.6
  [30578b45] URIParser v0.4.0
  [81def892] VersionParsing v1.1.3
  [2a0f44e3] Base64  [`@stdlib/Base64`]
  [ade2ca70] Dates  [`@stdlib/Dates`]
  [8bb1440f] DelimitedFiles  [`@stdlib/DelimitedFiles`]
  [8ba89e20] Distributed  [`@stdlib/Distributed`]
  [b77e0a4c] InteractiveUtils  [`@stdlib/InteractiveUtils`]
  [76f85450] LibGit2  [`@stdlib/LibGit2`]
  [8f399da3] Libdl  [`@stdlib/Libdl`]
  [37e2e46d] LinearAlgebra  [`@stdlib/LinearAlgebra`]
  [56ddb016] Logging  [`@stdlib/Logging`]
  [d6f4376e] Markdown  [`@stdlib/Markdown`]
  [a63ad114] Mmap  [`@stdlib/Mmap`]
  [44cfe95a] Pkg  [`@stdlib/Pkg`]
  [de0858da] Printf  [`@stdlib/Printf`]
  [3fa0cd96] REPL  [`@stdlib/REPL`]
  [9a3f8284] Random  [`@stdlib/Random`]
  [ea8e919c] SHA  [`@stdlib/SHA`]
  [9e88b42a] Serialization  [`@stdlib/Serialization`]
  [1a1011a3] SharedArrays  [`@stdlib/SharedArrays`]
  [6462fe0b] Sockets  [`@stdlib/Sockets`]
  [2f01184e] SparseArrays  [`@stdlib/SparseArrays`]
  [10745b16] Statistics  [`@stdlib/Statistics`]
  [8dfed614] Test  [`@stdlib/Test`]
  [cf7118a7] UUIDs  [`@stdlib/UUIDs`]
  [4ec0a83e] Unicode  [`@stdlib/Unicode`]
┌ Warning: Performing scalar operations on GPU arrays: This is very slow, consider disallowing these operations with `allowscalar(false)`
└ @ GPUArrays ~/.julia/packages/GPUArrays/fLiQ1/src/indexing.jl:16
Internal error: encountered unexpected error in runtime:
BoundsError(a=Array{Core.Compiler.BasicBlock, (32,)}[
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=1, last=7), preds=Array{Int64, (1,)}[32], succs=Array{Int64, (1,)}[2]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=8, last=14), preds=Array{Int64, (1,)}[1], succs=Array{Int64, (2,)}[5, 3]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=15, last=20), preds=Array{Int64, (1,)}[2], succs=Array{Int64, (1,)}[4]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=21, last=21), preds=Array{Int64, (1,)}[3], succs=Array{Int64, (1,)}[7]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=22, last=28), preds=Array{Int64, (1,)}[2], succs=Array{Int64, (1,)}[6]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=29, last=29), preds=Array{Int64, (1,)}[5], succs=Array{Int64, (1,)}[7]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=30, last=47), preds=Array{Int64, (2,)}[4, 6], succs=Array{Int64, (2,)}[9, 8]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=48, last=49), preds=Array{Int64, (1,)}[7], succs=Array{Int64, (0,)}[]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=50, last=50), preds=Array{Int64, (1,)}[7], succs=Array{Int64, (1,)}[10]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=51, last=52), preds=Array{Int64, (1,)}[9], succs=Array{Int64, (1,)}[11]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=53, last=53), preds=Array{Int64, (1,)}[10], succs=Array{Int64, (1,)}[12]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=54, last=54), preds=Array{Int64, (1,)}[11], succs=Array{Int64, (1,)}[13]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=55, last=56), preds=Array{Int64, (1,)}[12], succs=Array{Int64, (1,)}[14]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=57, last=65), preds=Array{Int64, (1,)}[13], succs=Array{Int64, (1,)}[15]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=66, last=71), preds=Array{Int64, (1,)}[14], succs=Array{Int64, (2,)}[17, 16]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=72, last=74), preds=Array{Int64, (1,)}[15], succs=Array{Int64, (0,)}[]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=75, last=82), preds=Array{Int64, (1,)}[15], succs=Array{Int64, (2,)}[19, 18]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=83, last=84), preds=Array{Int64, (1,)}[17], succs=Array{Int64, (0,)}[]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=85, last=85), preds=Array{Int64, (1,)}[17], succs=Array{Int64, (1,)}[20]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=86, last=87), preds=Array{Int64, (1,)}[19], succs=Array{Int64, (1,)}[21]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=88, last=88), preds=Array{Int64, (1,)}[20], succs=Array{Int64, (1,)}[22]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=89, last=89), preds=Array{Int64, (1,)}[21], succs=Array{Int64, (1,)}[23]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=90, last=90), preds=Array{Int64, (1,)}[22], succs=Array{Int64, (1,)}[24]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=91, last=92), preds=Array{Int64, (1,)}[23], succs=Array{Int64, (1,)}[25]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=93, last=93), preds=Array{Int64, (1,)}[24], succs=Array{Int64, (1,)}[26]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=94, last=94), preds=Array{Int64, (1,)}[25], succs=Array{Int64, (1,)}[27]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=95, last=95), preds=Array{Int64, (1,)}[26], succs=Array{Int64, (2,)}[29, 28]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=96, last=98), preds=Array{Int64, (1,)}[27], succs=Array{Int64, (0,)}[]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=99, last=102), preds=Array{Int64, (1,)}[27], succs=Array{Int64, (2,)}[31, 30]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=103, last=103), preds=Array{Int64, (1,)}[29], succs=Array{Int64, (1,)}[32]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=104, last=104), preds=Array{Int64, (1,)}[29], succs=Array{Int64, (1,)}[32]),
  Core.Compiler.BasicBlock(stmts=Core.Compiler.StmtRange(first=105, last=107), preds=Array{Int64, (2,)}[30, 31], succs=Array{Int64, (1,)}[1])], i=(0,))
rec_backtrace at /usr/local/src/conda/julia-1.0.3/src/stackwalk.c:94
record_backtrace at /usr/local/src/conda/julia-1.0.3/src/task.c:246
jl_throw at /usr/local/src/conda/julia-1.0.3/src/task.c:577
jl_bounds_error_ints at /usr/local/src/conda/julia-1.0.3/src/rtutils.c:187
getindex at ./array.jl:731
jfptr_getindex_1594 at /home/gilbert/anaconda3/envs/econda/lib/julia/sys.so (unknown line)
jl_apply_generic at /usr/local/src/conda/julia-1.0.3/src/gf.c:2184
replace_code_newstyle! at ./compiler/ssair/legacy.jl:80
optimize at ./compiler/optimize.jl:212
typeinf at ./compiler/typeinfer.jl:35
typeinf_ext at ./compiler/typeinfer.jl:574
typeinf_ext at ./compiler/typeinfer.jl:611
jfptr_typeinf_ext_1 at /home/gilbert/anaconda3/envs/econda/lib/julia/sys.so (unknown line)
jl_apply_generic at /usr/local/src/conda/julia-1.0.3/src/gf.c:2184
jl_apply at /usr/local/src/conda/julia-1.0.3/src/julia.h:1537 [inlined]
jl_apply_with_saved_exception_state at /usr/local/src/conda/julia-1.0.3/src/rtutils.c:257
jl_type_infer at /usr/local/src/conda/julia-1.0.3/src/gf.c:275
jl_compile_method_internal at /usr/local/src/conda/julia-1.0.3/src/gf.c:1786 [inlined]
jl_fptr_trampoline at /usr/local/src/conda/julia-1.0.3/src/gf.c:1830
jl_apply_generic at /usr/local/src/conda/julia-1.0.3/src/gf.c:2184
jl_apply at /usr/local/src/conda/julia-1.0.3/src/julia.h:1537 [inlined]
start_task at /usr/local/src/conda/julia-1.0.3/src/task.c:268
unknown function (ip: 0xffffffffffffffff)
getrf!: Error During Test at /home/gilbert/.julia/packages/CuArrays/wXQp8/test/solver.jl:44
  Got exception outside of a @test
  MethodError: no method matching LU(::Array{Float32,2}, ::Array{Int64,1}, ::Int64)
  Closest candidates are:
    LU(::AbstractArray{T,2}, !Matched::Array{Int32,1}, !Matched::Int32) where T at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/LinearAlgebra/src/lu.jl:17
  Stacktrace:
   [1] macro expansion at /home/gilbert/.julia/packages/CuArrays/wXQp8/test/solver.jl:50 [inlined]
   [2] macro expansion at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
   [3] macro expansion at /home/gilbert/.julia/packages/CuArrays/wXQp8/test/solver.jl:45 [inlined]
   [4] macro expansion at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1156 [inlined]
   [5] macro expansion at /home/gilbert/.julia/packages/CuArrays/wXQp8/test/solver.jl:14 [inlined]
   [6] macro expansion at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
   [7] top-level scope at /home/gilbert/.julia/packages/CuArrays/wXQp8/test/solver.jl:3
   [8] include at ./boot.jl:317 [inlined]
   [9] include_relative(::Module, ::String) at ./loading.jl:1044
   [10] include(::Module, ::String) at ./sysimg.jl:29
   [11] include(::String) at ./client.jl:392
   [12] macro expansion at /home/gilbert/.julia/packages/CuArrays/wXQp8/test/runtests.jl:25 [inlined]
   [13] macro expansion at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
   [14] top-level scope at /home/gilbert/.julia/packages/CuArrays/wXQp8/test/runtests.jl:20
   [15] include at ./boot.jl:317 [inlined]
   [16] include_relative(::Module, ::String) at ./loading.jl:1044
   [17] include(::Module, ::String) at ./sysimg.jl:29
   [18] include(::String) at ./client.jl:392
   [19] top-level scope at none:0
   [20] eval(::Module, ::Any) at ./boot.jl:319
   [21] exec_options(::Base.JLOptions) at ./client.jl:243
   [22] _start() at ./client.jl:425
getrf!: Error During Test at /home/gilbert/.julia/packages/CuArrays/wXQp8/test/solver.jl:44
  Got exception outside of a @test
  MethodError: no method matching LU(::Array{Float64,2}, ::Array{Int64,1}, ::Int64)
  Closest candidates are:
    LU(::AbstractArray{T,2}, !Matched::Array{Int32,1}, !Matched::Int32) where T at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/LinearAlgebra/src/lu.jl:17
  Stacktrace:
   [1] macro expansion at /home/gilbert/.julia/packages/CuArrays/wXQp8/test/solver.jl:50 [inlined]
   [2] macro expansion at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
   [3] macro expansion at /home/gilbert/.julia/packages/CuArrays/wXQp8/test/solver.jl:45 [inlined]
   [4] macro expansion at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1156 [inlined]
   [5] macro expansion at /home/gilbert/.julia/packages/CuArrays/wXQp8/test/solver.jl:14 [inlined]
   [6] macro expansion at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
   [7] top-level scope at /home/gilbert/.julia/packages/CuArrays/wXQp8/test/solver.jl:3
   [8] include at ./boot.jl:317 [inlined]
   [9] include_relative(::Module, ::String) at ./loading.jl:1044
   [10] include(::Module, ::String) at ./sysimg.jl:29
   [11] include(::String) at ./client.jl:392
   [12] macro expansion at /home/gilbert/.julia/packages/CuArrays/wXQp8/test/runtests.jl:25 [inlined]
   [13] macro expansion at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
   [14] top-level scope at /home/gilbert/.julia/packages/CuArrays/wXQp8/test/runtests.jl:20
   [15] include at ./boot.jl:317 [inlined]
   [16] include_relative(::Module, ::String) at ./loading.jl:1044
   [17] include(::Module, ::String) at ./sysimg.jl:29
   [18] include(::String) at ./client.jl:392
   [19] top-level scope at none:0
   [20] eval(::Module, ::Any) at ./boot.jl:319
   [21] exec_options(::Base.JLOptions) at ./client.jl:243
   [22] _start() at ./client.jl:425
getrf!: Error During Test at /home/gilbert/.julia/packages/CuArrays/wXQp8/test/solver.jl:44
  Got exception outside of a @test
  MethodError: no method matching LU(::Array{Complex{Float32},2}, ::Array{Int64,1}, ::Int64)
  Closest candidates are:
    LU(::AbstractArray{T,2}, !Matched::Array{Int32,1}, !Matched::Int32) where T at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/LinearAlgebra/src/lu.jl:17
  Stacktrace:
   [1] macro expansion at /home/gilbert/.julia/packages/CuArrays/wXQp8/test/solver.jl:50 [inlined]
   [2] macro expansion at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
   [3] macro expansion at /home/gilbert/.julia/packages/CuArrays/wXQp8/test/solver.jl:45 [inlined]
   [4] macro expansion at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1156 [inlined]
   [5] macro expansion at /home/gilbert/.julia/packages/CuArrays/wXQp8/test/solver.jl:14 [inlined]
   [6] macro expansion at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
   [7] top-level scope at /home/gilbert/.julia/packages/CuArrays/wXQp8/test/solver.jl:3
   [8] include at ./boot.jl:317 [inlined]
   [9] include_relative(::Module, ::String) at ./loading.jl:1044
   [10] include(::Module, ::String) at ./sysimg.jl:29
   [11] include(::String) at ./client.jl:392
   [12] macro expansion at /home/gilbert/.julia/packages/CuArrays/wXQp8/test/runtests.jl:25 [inlined]
   [13] macro expansion at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
   [14] top-level scope at /home/gilbert/.julia/packages/CuArrays/wXQp8/test/runtests.jl:20
   [15] include at ./boot.jl:317 [inlined]
   [16] include_relative(::Module, ::String) at ./loading.jl:1044
   [17] include(::Module, ::String) at ./sysimg.jl:29
   [18] include(::String) at ./client.jl:392
   [19] top-level scope at none:0
   [20] eval(::Module, ::Any) at ./boot.jl:319
   [21] exec_options(::Base.JLOptions) at ./client.jl:243
   [22] _start() at ./client.jl:425
getrf!: Error During Test at /home/gilbert/.julia/packages/CuArrays/wXQp8/test/solver.jl:44
  Got exception outside of a @test
  MethodError: no method matching LU(::Array{Complex{Float64},2}, ::Array{Int64,1}, ::Int64)
  Closest candidates are:
    LU(::AbstractArray{T,2}, !Matched::Array{Int32,1}, !Matched::Int32) where T at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/LinearAlgebra/src/lu.jl:17
  Stacktrace:
   [1] macro expansion at /home/gilbert/.julia/packages/CuArrays/wXQp8/test/solver.jl:50 [inlined]
   [2] macro expansion at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
   [3] macro expansion at /home/gilbert/.julia/packages/CuArrays/wXQp8/test/solver.jl:45 [inlined]
   [4] macro expansion at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1156 [inlined]
   [5] macro expansion at /home/gilbert/.julia/packages/CuArrays/wXQp8/test/solver.jl:14 [inlined]
   [6] macro expansion at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
   [7] top-level scope at /home/gilbert/.julia/packages/CuArrays/wXQp8/test/solver.jl:3
   [8] include at ./boot.jl:317 [inlined]
   [9] include_relative(::Module, ::String) at ./loading.jl:1044
   [10] include(::Module, ::String) at ./sysimg.jl:29
   [11] include(::String) at ./client.jl:392
   [12] macro expansion at /home/gilbert/.julia/packages/CuArrays/wXQp8/test/runtests.jl:25 [inlined]
   [13] macro expansion at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Test/src/Test.jl:1083 [inlined]
   [14] top-level scope at /home/gilbert/.julia/packages/CuArrays/wXQp8/test/runtests.jl:20
   [15] include at ./boot.jl:317 [inlined]
   [16] include_relative(::Module, ::String) at ./loading.jl:1044
   [17] include(::Module, ::String) at ./sysimg.jl:29
   [18] include(::String) at ./client.jl:392
   [19] top-level scope at none:0
   [20] eval(::Module, ::Any) at ./boot.jl:319
   [21] exec_options(::Base.JLOptions) at ./client.jl:243
   [22] _start() at ./client.jl:425
[ Info: Testing CUDNN 7.4.1
[ Info: Testing ForwardDiff integration
Total GPU memory usage: 6.26% (687.813 MiB/10.729 GiB)
CuArrays.jl pool usage: 0.49% (381.758 KiB in use by 182 buffers, 3.026 MiB idle)
 ────────────────────────────────────────────────
                                   Time
                           ──────────────────────
     Tot / % measured:           359s / 0.09%

 Section           ncalls     time   %tot     avg
 ────────────────────────────────────────────────
 pooled alloc       9.20k    177ms  56.4%  19.2μs
   1 try alloc      1.07k    173ms  55.1%   162μs
 background task        9    154ms  48.9%  17.1ms
   reclaim              9   10.3ms  3.26%  1.14ms
   scan                 9   11.3μs  0.00%  1.26μs
 ────────────────────────────────────────────────
Test Summary:                          | Pass  Error  Total
CuArrays                               | 4443      4   4447
  GPUArrays test suite                 | 1099          1099
  Memory                               |    5             5
  Array                                |   22            22
  Adapt                                |    2             2
  Broadcast                            |   18            18
  Cufunc                               |    8             8
  Ref Broadcast                        |    1             1
  Broadcast Fix                        |    4             4
  Reduce                               |    6             6
  0D                                   |    2             2
  SubArray                             |   20            20
  reshape                              |    1             1
  LinearAlgebra.triu! with diagonal -2 |    1             1
  LinearAlgebra.triu! with diagonal -1 |    1             1
  LinearAlgebra.triu! with diagonal 0  |    1             1
  LinearAlgebra.triu! with diagonal 1  |    1             1
  LinearAlgebra.triu! with diagonal 2  |    1             1
  LinearAlgebra.tril! with diagonal -2 |    1             1
  LinearAlgebra.tril! with diagonal -1 |    1             1
  LinearAlgebra.tril! with diagonal 0  |    1             1
  LinearAlgebra.tril! with diagonal 1  |    1             1
  LinearAlgebra.tril! with diagonal 2  |    1             1
  Utilities                            |    2             2
  accumulate                           |    8             8
  logical indexing                     |   15            15
  generic fallbacks                    |   11            11
  reverse                              |    6             6
  permutedims                          |    2             2
  CUBLAS                               | 1184          1184
  CURAND                               |   99            99
  CUFFT                                |  150           150
  CUSPARSE                             | 1229          1229
  CUSOLVER                             |  289      4    293
    elty = Float32                     |   72      1     73
      Cholesky (po)                    |    8             8
      getrf!                           |           1      1
      getrs!                           |    3             3
      geqrf!                           |    1             1
      ormqr!                           |    4             4
      orgqr!                           |    2             2
      sytrf!                           |    4             4
      gebrd!                           |    4             4
      syevd!                           |    3             3
      sygvd!                           |    5             5
      syevj!                           |    4             4
      svd with QRAlgorithm method      |    7             7
      svd with QRAlgorithm method      |    1             1
      svd with JacobiAlgorithm method  |    7             7
      svd with JacobiAlgorithm method  |    7             7
      qr                               |   12            12
    elty = Float64                     |   72      1     73
      Cholesky (po)                    |    8             8
      getrf!                           |           1      1
      getrs!                           |    3             3
      geqrf!                           |    1             1
      ormqr!                           |    4             4
      orgqr!                           |    2             2
      sytrf!                           |    4             4
      gebrd!                           |    4             4
      syevd!                           |    3             3
      sygvd!                           |    5             5
      syevj!                           |    4             4
      svd with QRAlgorithm method      |    7             7
      svd with QRAlgorithm method      |    1             1
      svd with JacobiAlgorithm method  |    7             7
      svd with JacobiAlgorithm method  |    7             7
      qr                               |   12            12
    elty = Complex{Float32}            |   72      1     73
      Cholesky (po)                    |    8             8
      getrf!                           |           1      1
      getrs!                           |    3             3
      geqrf!                           |    1             1
      ormqr!                           |    4             4
      orgqr!                           |    2             2
      sytrf!                           |    4             4
      gebrd!                           |    4             4
      syevd!                           |    3             3
      sygvd!                           |    5             5
      syevj!                           |    4             4
      svd with QRAlgorithm method      |    7             7
      svd with QRAlgorithm method      |    1             1
      svd with JacobiAlgorithm method  |    7             7
      svd with JacobiAlgorithm method  |    7             7
      qr                               |   12            12
    elty = Complex{Float64}            |   72      1     73
      Cholesky (po)                    |    8             8
      getrf!                           |           1      1
      getrs!                           |    3             3
      geqrf!                           |    1             1
      ormqr!                           |    4             4
      orgqr!                           |    2             2
      sytrf!                           |    4             4
      gebrd!                           |    4             4
      syevd!                           |    3             3
      sygvd!                           |    5             5
      syevj!                           |    4             4
      svd with QRAlgorithm method      |    7             7
      svd with QRAlgorithm method      |    1             1
      svd with JacobiAlgorithm method  |    7             7
      svd with JacobiAlgorithm method  |    7             7
      qr                               |   12            12
  CUSPARSE + CUSOLVER                  |   84            84
  CUDNN                                |   70            70
  ForwardDiff                          |   96            96
ERROR: LoadError: Some tests did not pass: 4443 passed, 0 failed, 4 errored, 0 broken.
in expression starting at /home/gilbert/.julia/packages/CuArrays/wXQp8/test/runtests.jl:18
ERROR: Package CuArrays errored during testing
Stacktrace:
 [1] pkgerror(::String, ::Vararg{String,N} where N) at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Pkg/src/Types.jl:120
 [2] #test#65(::Bool, ::Function, ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Pkg/src/Operations.jl:1357
 [3] #test at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:0 [inlined]
 [4] #test#42(::Bool, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:273
 [5] test at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:258 [inlined]
 [6] #test#41 at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:255 [inlined]
 [7] test at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:255 [inlined]
 [8] #test#40 at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:254 [inlined]
 [9] test at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:254 [inlined]
 [10] #test#39 at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:253 [inlined]
 [11] test(::String) at /home/conda/feedstock_root/build_artifacts/julia_1548684429855/work/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:253
 [12] top-level scope at none:0

John_G · September 23, 2019, 3:31pm

Additionally I haven’t been able to get CuArrays to work properly, and I have yet to find a Flux tutorial that I can run succesfully in its entirety. I am running my code in a conda environment on a server with a GPU without root access, and I’ve been able to run deep learning models using Pytorch and fastai without any problems in the same conda environment.

The server is running on Ubuntu 16.04.6 LTS (GNU/Linux 4.15.0-58-generic x86_64).

This is information I get on the GPU when I run lshw -C display from the terminal:

(GPU Information: Click to expand)

description: VGA compatible controller
       product: GV102
       vendor: NVIDIA Corporation
       physical id: 0
       bus info: pci@0000:01:00.0
       version: a1
       width: 64 bits
       clock: 33MHz
       capabilities: vga_controller bus_master cap_list rom
       configuration: driver=nvidia latency=0
       resources: irq:126 memory:a3000000-a3ffffff memory:90000000-9fffffff memory:a0000000-a1ffffff ioport:3000(size=128)memory:c0000-dffff
WARNING: output may be incomplete or inaccurate, you should run this program as super-user.

I am running Julia Version 1.0.3 (conda-forge-julia release). These is the output from running using Pkg; Pkg.installed():

(Pkg info: Click to expand)

Dict{String,Union{Nothing, VersionNumber}} with 12 entries:
  "CSV"            => v"0.5.12"
  "Missings"       => v"0.4.2"
  "Metalhead"      => v"0.3.0"
  "DataFramesMeta" => v"0.5.0"
  "StatsBase"      => v"0.32.0"
  "CuArrays"       => v"1.2.1"
  "IJulia"         => v"1.20.0"
  "Flux"           => v"0.9.0"
  "CUDAnative"     => v"2.3.1"
  "DataFrames"     => v"0.19.4"
  "FilePaths"      => v"0.8.0"
  "Pandas"         => v"1.3.0"

I don’t have a ton of experience with Julia, but I am really interested in getting setup using Julia for some of my deep learning projects. As the above error messages are unintuitive to me, any help would be very much appreciated.

(Note: I split this message up into parts due to the character limit on discourse)

vitskvara · October 2, 2019, 4:22pm

Consider updating Julia to at least 1.1, I had similar issues that disappeared with Julia version update.