I have a file test.jl of four lines of Julia code
using CUDA
A = CuArray(rand(1000,1000));
B = CuArray(rand(1000,1000));
C = A * B;
After running the command
cuda-memcheck julia test.jl >> test.jl.log
I have got so many lines in the report, which indicates some serious errors I don’t understand.
What is going on?
content of test.jl.log
========= CUDA-MEMCHECK
========= This tool is deprecated and will be removed in a future release of the CUDA toolkit
========= Please use the compute-sanitizer tool as a drop-in replacement
========= Internal Memcheck Warning: Detected use of unsupported CUDA memory pools. Please use the compute-sanitizer tool instead.
=========
========= Program hit CUDA_ERROR_INVALID_VALUE (error 1) due to "invalid argument" on CUDA API call to cuPointerGetAttribute.
========= Saved host backtrace up to driver entry point at error
========= Host Frame:/lib/x86_64-linux-gnu/libcuda.so.1 [0x232f1c]
========= Host Frame:[0x7fa6e96c0550]
========= Host Frame:[0x7fa6e96c07bf]
========= Host Frame:[0x7fa6e96c1a4c]
========= Host Frame:[0x7fa6e96c1ca1]
========= Host Frame:/home/dabajabaza/julia/bin/../lib/julia/libjulia-internal.so.1 (jl_apply_generic + 0x1fa) [0xc4e0a]
========= Host Frame:/home/dabajabaza/julia/bin/../lib/julia/libjulia-internal.so.1 [0xe3e96]
========= Host Frame:/home/dabajabaza/julia/bin/../lib/julia/libjulia-internal.so.1 [0xe390e]
========= Host Frame:/home/dabajabaza/julia/bin/../lib/julia/libjulia-internal.so.1 [0xe46d2]
========= Host Frame:/home/dabajabaza/julia/bin/../lib/julia/libjulia-internal.so.1 [0xe52f8]
========= Host Frame:/home/dabajabaza/julia/bin/../lib/julia/libjulia-internal.so.1 [0x1027a4]
========= Host Frame:/home/dabajabaza/julia/bin/../lib/julia/libjulia-internal.so.1 [0x1029e5]
========= Host Frame:/home/dabajabaza/julia/bin/../lib/julia/libjulia-internal.so.1 [0x1029e5]
========= Host Frame:/home/dabajabaza/julia/bin/../lib/julia/libjulia-internal.so.1 (jl_toplevel_eval_in + 0xaa) [0x10462a]
========= Host Frame:/home/dabajabaza/julia/lib/julia/sys.so [0x115a83b]
========= Host Frame:/home/dabajabaza/julia/bin/../lib/julia/libjulia-internal.so.1 (jl_apply_generic + 0x1fa) [0xc4e0a]
========= Host Frame:/home/dabajabaza/julia/lib/julia/sys.so [0x124a35b]
========= Host Frame:/home/dabajabaza/julia/lib/julia/sys.so [0xd67c16]
========= Host Frame:/home/dabajabaza/julia/bin/../lib/julia/libjulia-internal.so.1 (jl_apply_generic + 0x1fa) [0xc4e0a]
========= Host Frame:/home/dabajabaza/julia/lib/julia/sys.so [0x12d064c]
========= Host Frame:/home/dabajabaza/julia/lib/julia/sys.so [0xd8a0f8]
========= Host Frame:/home/dabajabaza/julia/lib/julia/sys.so [0xd8a269]
========= Host Frame:/home/dabajabaza/julia/bin/../lib/julia/libjulia-internal.so.1 (jl_apply_generic + 0x1fa) [0xc4e0a]
========= Host Frame:/home/dabajabaza/julia/bin/../lib/julia/libjulia-internal.so.1 [0x1282d6]
========= Host Frame:/home/dabajabaza/julia/bin/../lib/julia/libjulia-internal.so.1 (jl_repl_entrypoint + 0x8d) [0x128c7d]
========= Host Frame:julia (main + 0x9) [0x7d9]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 [0x29fd0]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 (__libc_start_main + 0x7d) [0x2a07d]
========= Host Frame:julia [0x809]
=========
========= Host API memory access error at host access to 0x302000000 of size 8000000 bytes
========= Invalid start allocation on access by cudaMemcopy destination.
========= Saved host backtrace up to driver entry point at error
========= Host Frame:/lib/x86_64-linux-gnu/libcuda.so.1 [0x1fcc29]
========= Host Frame:[0x7fa6e96c0085]
========= Host Frame:[0x7fa6e96c02a0]
========= Host Frame:[0x7fa6e96c107a]
========= Host Frame:[0x7fa6e96c1a4c]
========= Host Frame:[0x7fa6e96c1ca1]
========= Host Frame:/home/dabajabaza/julia/bin/../lib/julia/libjulia-internal.so.1 (jl_apply_generic + 0x1fa) [0xc4e0a]
========= Host Frame:/home/dabajabaza/julia/bin/../lib/julia/libjulia-internal.so.1 [0xe3e96]
========= Host Frame:/home/dabajabaza/julia/bin/../lib/julia/libjulia-internal.so.1 [0xe390e]
========= Host Frame:/home/dabajabaza/julia/bin/../lib/julia/libjulia-internal.so.1 [0xe46d2]
========= Host Frame:/home/dabajabaza/julia/bin/../lib/julia/libjulia-internal.so.1 [0xe52f8]
========= Host Frame:/home/dabajabaza/julia/bin/../lib/julia/libjulia-internal.so.1 [0x1027a4]
========= Host Frame:/home/dabajabaza/julia/bin/../lib/julia/libjulia-internal.so.1 [0x1029e5]
========= Host Frame:/home/dabajabaza/julia/bin/../lib/julia/libjulia-internal.so.1 [0x1029e5]
========= Host Frame:/home/dabajabaza/julia/bin/../lib/julia/libjulia-internal.so.1 (jl_toplevel_eval_in + 0xaa) [0x10462a]
========= Host Frame:/home/dabajabaza/julia/lib/julia/sys.so [0x115a83b]
========= Host Frame:/home/dabajabaza/julia/bin/../lib/julia/libjulia-internal.so.1 (jl_apply_generic + 0x1fa) [0xc4e0a]
========= Host Frame:/home/dabajabaza/julia/lib/julia/sys.so [0x124a35b]
========= Host Frame:/home/dabajabaza/julia/lib/julia/sys.so [0xd67c16]
========= Host Frame:/home/dabajabaza/julia/bin/../lib/julia/libjulia-internal.so.1 (jl_apply_generic + 0x1fa) [0xc4e0a]
========= Host Frame:/home/dabajabaza/julia/lib/julia/sys.so [0x12d064c]
========= Host Frame:/home/dabajabaza/julia/lib/julia/sys.so [0xd8a0f8]
========= Host Frame:/home/dabajabaza/julia/lib/julia/sys.so [0xd8a269]
========= Host Frame:/home/dabajabaza/julia/bin/../lib/julia/libjulia-internal.so.1 (jl_apply_generic + 0x1fa) [0xc4e0a]
========= Host Frame:/home/dabajabaza/julia/bin/../lib/julia/libjulia-internal.so.1 [0x1282d6]
========= Host Frame:/home/dabajabaza/julia/bin/../lib/julia/libjulia-internal.so.1 (jl_repl_entrypoint + 0x8d) [0x128c7d]
========= Host Frame:julia (main + 0x9) [0x7d9]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 [0x29fd0]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 (__libc_start_main + 0x7d) [0x2a07d]
========= Host Frame:julia [0x809]
=========
...
...
...
========= Program hit CUDA_ERROR_LAUNCH_FAILED (error 719) due to "unspecified launch failure" on CUDA API call to cuModuleUnload.
========= Saved host backtrace up to driver entry point at error
========= Host Frame:/lib/x86_64-linux-gnu/libcuda.so.1 [0x245d97]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a324e]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a539c]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a621f]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x2497a32]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x2498295]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 [0x454e5]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 (on_exit + 0x0) [0x45660]
========= Host Frame:julia (_start + 0x0) [0x7e0]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 [0x29fd0]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 (__libc_start_main + 0x7d) [0x2a07d]
========= Host Frame:julia [0x809]
=========
========= Program hit CUDA_ERROR_LAUNCH_FAILED (error 719) due to "unspecified launch failure" on CUDA API call to cuModuleUnload.
========= Saved host backtrace up to driver entry point at error
========= Host Frame:/lib/x86_64-linux-gnu/libcuda.so.1 [0x245d97]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a324e]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a539c]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a621f]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x2497a32]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x2498295]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 [0x454e5]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 (on_exit + 0x0) [0x45660]
========= Host Frame:julia (_start + 0x0) [0x7e0]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 [0x29fd0]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 (__libc_start_main + 0x7d) [0x2a07d]
========= Host Frame:julia [0x809]
=========
========= Program hit CUDA_ERROR_LAUNCH_FAILED (error 719) due to "unspecified launch failure" on CUDA API call to cuModuleUnload.
========= Saved host backtrace up to driver entry point at error
========= Host Frame:/lib/x86_64-linux-gnu/libcuda.so.1 [0x245d97]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a324e]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a539c]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a621f]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x2497a32]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x2498295]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 [0x454e5]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 (on_exit + 0x0) [0x45660]
========= Host Frame:julia (_start + 0x0) [0x7e0]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 [0x29fd0]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 (__libc_start_main + 0x7d) [0x2a07d]
========= Host Frame:julia [0x809]
=========
========= Program hit CUDA_ERROR_LAUNCH_FAILED (error 719) due to "unspecified launch failure" on CUDA API call to cuModuleUnload.
========= Saved host backtrace up to driver entry point at error
========= Host Frame:/lib/x86_64-linux-gnu/libcuda.so.1 [0x245d97]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a324e]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a539c]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a621f]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x2497a32]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x2498295]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 [0x454e5]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 (on_exit + 0x0) [0x45660]
========= Host Frame:julia (_start + 0x0) [0x7e0]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 [0x29fd0]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 (__libc_start_main + 0x7d) [0x2a07d]
========= Host Frame:julia [0x809]
=========
========= Program hit CUDA_ERROR_LAUNCH_FAILED (error 719) due to "unspecified launch failure" on CUDA API call to cuModuleUnload.
========= Saved host backtrace up to driver entry point at error
========= Host Frame:/lib/x86_64-linux-gnu/libcuda.so.1 [0x245d97]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a324e]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a539c]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a621f]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x2497a32]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x2498295]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 [0x454e5]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 (on_exit + 0x0) [0x45660]
========= Host Frame:julia (_start + 0x0) [0x7e0]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 [0x29fd0]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 (__libc_start_main + 0x7d) [0x2a07d]
========= Host Frame:julia [0x809]
=========
========= Program hit CUDA_ERROR_LAUNCH_FAILED (error 719) due to "unspecified launch failure" on CUDA API call to cuModuleUnload.
========= Saved host backtrace up to driver entry point at error
========= Host Frame:/lib/x86_64-linux-gnu/libcuda.so.1 [0x245d97]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a324e]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a539c]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a621f]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x2497a32]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x2498295]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 [0x454e5]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 (on_exit + 0x0) [0x45660]
========= Host Frame:julia (_start + 0x0) [0x7e0]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 [0x29fd0]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 (__libc_start_main + 0x7d) [0x2a07d]
========= Host Frame:julia [0x809]
=========
========= Program hit CUDA_ERROR_LAUNCH_FAILED (error 719) due to "unspecified launch failure" on CUDA API call to cuModuleUnload.
========= Saved host backtrace up to driver entry point at error
========= Host Frame:/lib/x86_64-linux-gnu/libcuda.so.1 [0x245d97]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a324e]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a539c]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a621f]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x2497a32]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x2498295]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 [0x454e5]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 (on_exit + 0x0) [0x45660]
========= Host Frame:julia (_start + 0x0) [0x7e0]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 [0x29fd0]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 (__libc_start_main + 0x7d) [0x2a07d]
========= Host Frame:julia [0x809]
=========
========= Program hit CUDA_ERROR_LAUNCH_FAILED (error 719) due to "unspecified launch failure" on CUDA API call to cuModuleUnload.
========= Saved host backtrace up to driver entry point at error
========= Host Frame:/lib/x86_64-linux-gnu/libcuda.so.1 [0x245d97]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a324e]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a539c]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a621f]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x2497a32]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x2498295]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 [0x454e5]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 (on_exit + 0x0) [0x45660]
========= Host Frame:julia (_start + 0x0) [0x7e0]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 [0x29fd0]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 (__libc_start_main + 0x7d) [0x2a07d]
========= Host Frame:julia [0x809]
=========
========= Program hit CUDA_ERROR_LAUNCH_FAILED (error 719) due to "unspecified launch failure" on CUDA API call to cuModuleUnload.
========= Saved host backtrace up to driver entry point at error
========= Host Frame:/lib/x86_64-linux-gnu/libcuda.so.1 [0x245d97]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a324e]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a539c]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a621f]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x2497a32]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x2498295]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 [0x454e5]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 (on_exit + 0x0) [0x45660]
========= Host Frame:julia (_start + 0x0) [0x7e0]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 [0x29fd0]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 (__libc_start_main + 0x7d) [0x2a07d]
========= Host Frame:julia [0x809]
=========
========= Program hit CUDA_ERROR_LAUNCH_FAILED (error 719) due to "unspecified launch failure" on CUDA API call to cuModuleUnload.
========= Saved host backtrace up to driver entry point at error
========= Host Frame:/lib/x86_64-linux-gnu/libcuda.so.1 [0x245d97]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a324e]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a539c]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a621f]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x2497a32]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x2498295]333
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 [0x454e5]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 (on_exit + 0x0) [0x45660]
========= Host Frame:julia (_start + 0x0) [0x7e0]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 [0x29fd0]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 (__libc_start_main + 0x7d) [0x2a07d]
========= Host Frame:julia [0x809]
=========
========= Program hit CUDA_ERROR_LAUNCH_FAILED (error 719) due to "unspecified launch failure" on CUDA API call to cuModuleUnload.
========= Saved host backtrace up to driver entry point at error
========= Host Frame:/lib/x86_64-linux-gnu/libcuda.so.1 [0x245d97]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a324e]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a539c]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a621f]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x2497a32]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x2498295]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 [0x454e5]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 (on_exit + 0x0) [0x45660]
========= Host Frame:julia (_start + 0x0) [0x7e0]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 [0x29fd0]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 (__libc_start_main + 0x7d) [0x2a07d]
========= Host Frame:julia [0x809]
=========
========= Program hit CUDA_ERROR_LAUNCH_FAILED (error 719) due to "unspecified launch failure" on CUDA API call to cuModuleUnload.
========= Saved host backtrace up to driver entry point at error
========= Host Frame:/lib/x86_64-linux-gnu/libcuda.so.1 [0x245d97]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a324e]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a539c]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a621f]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x2497a32]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x2498295]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 [0x454e5]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 (on_exit + 0x0) [0x45660]
========= Host Frame:julia (_start + 0x0) [0x7e0]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 [0x29fd0]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 (__libc_start_main + 0x7d) [0x2a07d]
========= Host Frame:julia [0x809]
=========
========= Program hit CUDA_ERROR_LAUNCH_FAILED (error 719) due to "unspecified launch failure" on CUDA API call to cuModuleUnload.
========= Saved host backtrace up to driver entry point at error
========= Host Frame:/lib/x86_64-linux-gnu/libcuda.so.1 [0x245d97]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a324e]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a539c]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a621f]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x2497a32]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x2498295]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 [0x454e5]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 (on_exit + 0x0) [0x45660]
========= Host Frame:julia (_start + 0x0) [0x7e0]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 [0x29fd0]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 (__libc_start_main + 0x7d) [0x2a07d]
========= Host Frame:julia [0x809]
=========
========= Program hit CUDA_ERROR_LAUNCH_FAILED (error 719) due to "unspecified launch failure" on CUDA API call to cuModuleUnload.
========= Saved host backtrace up to driver entry point at error
========= Host Frame:/lib/x86_64-linux-gnu/libcuda.so.1 [0x245d97]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a324e]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a539c]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a621f]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x2497a32]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x2498295]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 [0x454e5]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 (on_exit + 0x0) [0x45660]
========= Host Frame:julia (_start + 0x0) [0x7e0]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 [0x29fd0]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 (__libc_start_main + 0x7d) [0x2a07d]
========= Host Frame:julia [0x809]
=========
========= Program hit CUDA_ERROR_LAUNCH_FAILED (error 719) due to "unspecified launch failure" on CUDA API call to cuModuleUnload.
========= Saved host backtrace up to driver entry point at error
========= Host Frame:/lib/x86_64-linux-gnu/libcuda.so.1 [0x245d97]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a324e]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a539c]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a621f]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x2497a32]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x2498295]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 [0x454e5]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 (on_exit + 0x0) [0x45660]
========= Host Frame:julia (_start + 0x0) [0x7e0]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 [0x29fd0]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 (__libc_start_main + 0x7d) [0x2a07d]
========= Host Frame:julia [0x809]
=========
========= Program hit CUDA_ERROR_LAUNCH_FAILED (error 719) due to "unspecified launch failure" on CUDA API call to cuModuleUnload.
========= Saved host backtrace up to driver entry point at error
========= Host Frame:/lib/x86_64-linux-gnu/libcuda.so.1 [0x245d97]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a324e]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a539c]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a621f]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x2497a32]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x2498295]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 [0x454e5]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 (on_exit + 0x0) [0x45660]
========= Host Frame:julia (_start + 0x0) [0x7e0]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 [0x29fd0]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 (__libc_start_main + 0x7d) [0x2a07d]
========= Host Frame:julia [0x809]
=========
========= Program hit CUDA_ERROR_LAUNCH_FAILED (error 719) due to "unspecified launch failure" on CUDA API call to cuModuleUnload.
========= Saved host backtrace up to driver entry point at error
========= Host Frame:/lib/x86_64-linux-gnu/libcuda.so.1 [0x245d97]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a324e]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a539c]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a621f]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x2497a32]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x2498295]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 [0x454e5]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 (on_exit + 0x0) [0x45660]
========= Host Frame:julia (_start + 0x0) [0x7e0]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 [0x29fd0]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 (__libc_start_main + 0x7d) [0x2a07d]
========= Host Frame:julia [0x809]
=========
========= Program hit CUDA_ERROR_LAUNCH_FAILED (error 719) due to "unspecified launch failure" on CUDA API call to cuModuleUnload.
========= Saved host backtrace up to driver entry point at error
========= Host Frame:/lib/x86_64-linux-gnu/libcuda.so.1 [0x245d97]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a324e]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a539c]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x24a621f]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x2497a32]
========= Host Frame:/home/dabajabaza/.julia/artifacts/913584335ab836f9781a0325178d0949c193f50b/lib/libcublasLt.so [0x2498295]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 [0x454e5]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 (on_exit + 0x0) [0x45660]
========= Host Frame:julia (_start + 0x0) [0x7e0]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 [0x29fd0]
========= Host Frame:/lib/x86_64-linux-gnu/libc.so.6 (__libc_start_main + 0x7d) [0x2a07d]
========= Host Frame:julia [0x809]
=========
========= ERROR SUMMARY: 1361 errors