# CUDA.jl mystery : VSCode + Julia extension works fine but commandline run fails

**URL:** <https://discourse.julialang.org/t/cuda-jl-mystery-vscode-julia-extension-works-fine-but-commandline-run-fails/84524>\
**Category:** GPU\
**Tags:** question\
**Created:** [July 20, 2022, 10:18am UTC](https://discourse.julialang.org/t/cuda-jl-mystery-vscode-julia-extension-works-fine-but-commandline-run-fails/84524 "2022-07-20T10:18:02Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Lian\_Yunlong](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lian_yunlong/32/14437_2.png) [@Lian\_Yunlong](https://discourse.julialang.org/u/Lian_Yunlong)\
**Post date:** [July 20, 2022, 10:18am UTC](https://discourse.julialang.org/t/cuda-jl-mystery-vscode-julia-extension-works-fine-but-commandline-run-fails/84524/1 "2022-07-20T10:18:02Z")

</div>

Hi,  
I have encountered some mysterious issues with CUDA.jl. When I run the code from within the file (in VSCode + Julia extension), everything seems fine and I got the expected answer. But if I run the same code in command line, I got the following error:

```julia
  Activating project at `~/jianguoyun/Nutstore/RigorousCoupledWaveAnalysis.jl-master`
eigenmodes:
etm_propagate:
ERROR: LoadError: CUBLASError: an invalid value was used as an argument (code 7, CUBLAS_STATUS_INVALID_VALUE)
Stacktrace:
  [1] throw_api_error(res::CUDA.CUBLAS.cublasStatus_t)
    @ CUDA.CUBLAS ~/.julia/packages/CUDA/DfvRa/lib/cublas/error.jl:50
  [2] macro expansion
    @ ~/.julia/packages/CUDA/DfvRa/lib/cublas/error.jl:63 [inlined]
  [3] cublasZgemv_v2(handle::Ptr{Nothing}, trans::Char, m::Int64, n::Int64, alpha::Bool, A::CUDA.CuArray{ComplexF64, 2, CUDA.Mem.DeviceBuffer}, lda::Int64, x::CUDA.CuArray{ComplexF64, 1, CUDA.Mem.DeviceBuffer}, incx::Int64, beta::Bool, y::CUDA.CuArray{ComplexF64, 1, CUDA.Mem.DeviceBuffer}, incy::Int64)
    @ CUDA.CUBLAS ~/.julia/packages/CUDA/DfvRa/lib/utils/call.jl:26
  [4] gemv!
    @ ~/.julia/packages/CUDA/DfvRa/lib/cublas/wrappers.jl:331 [inlined]
  [5] gemv_dispatch!(Y::CUDA.CuArray{ComplexF64, 1, CUDA.Mem.DeviceBuffer}, A::CUDA.CuArray{ComplexF64, 2, CUDA.Mem.DeviceBuffer}, B::CUDA.CuArray{ComplexF64, 1, CUDA.Mem.DeviceBuffer}, alpha::Bool, beta::Bool)
    @ CUDA.CUBLAS ~/.julia/packages/CUDA/DfvRa/lib/cublas/linalg.jl:179
  [6] mul!
    @ ~/.julia/packages/CUDA/DfvRa/lib/cublas/linalg.jl:188 [inlined]
  [7] mul!
    @ ~/julia/share/julia/stdlib/v1.7/LinearAlgebra/src/matmul.jl:275 [inlined]
  [8] *(A::CUDA.CuArray{ComplexF64, 2, CUDA.Mem.DeviceBuffer}, x::CUDA.CuArray{ComplexF64, 1, CUDA.Mem.DeviceBuffer})
    @ LinearAlgebra ~/julia/share/julia/stdlib/v1.7/LinearAlgebra/src/matmul.jl:51
  [9] etm_propagate_gpu(sup::RigorousCoupledWaveAnalysis.Common.Halfspace, sub::RigorousCoupledWaveAnalysis.Common.Halfspace, ems_gpu::Vector{RigorousCoupledWaveAnalysis.Common.Eigenmodes}, ψin::CUDA.CuArray{ComplexF64, 1, CUDA.Mem.DeviceBuffer}, get_r::Bool)
    @ RigorousCoupledWaveAnalysis.ETM ~/jianguoyun/Nutstore/RigorousCoupledWaveAnalysis.jl-master/src/ETM/ETM.jl:133
 [10] etm_propagate
    @ ~/jianguoyun/Nutstore/RigorousCoupledWaveAnalysis.jl-master/src/ETM/ETM.jl:43 [inlined]
 [11] etm_propagate(sup::RigorousCoupledWaveAnalysis.Common.Halfspace, sub::RigorousCoupledWaveAnalysis.Common.Halfspace, em::Vector{RigorousCoupledWaveAnalysis.Common.Eigenmodes}, ψin::CUDA.CuArray{ComplexF64, 1, CUDA.Mem.DeviceBuffer}, grd::RigorousCoupledWaveAnalysis.Common.RCWAGrid)
    @ RigorousCoupledWaveAnalysis.ETM ~/jianguoyun/Nutstore/RigorousCoupledWaveAnalysis.jl-master/src/ETM/ETM.jl:38
 [12] top-level scope
    @ ~/jianguoyun/Nutstore/RigorousCoupledWaveAnalysis.jl-master/examples/test.augel2018.jl:56
in expression starting at /home/dabajabaza/jianguoyun/Nutstore/RigorousCoupledWaveAnalysis.jl-master/examples/test.augel2018.jl:56

```

Anyone knows what is going wrong?

---

<div class="post-metadata">

**Author:** ![Lian\_Yunlong](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lian_yunlong/32/14437_2.png) [@Lian\_Yunlong](https://discourse.julialang.org/u/Lian_Yunlong)\
**Post date:** [July 20, 2022, 10:19am UTC](https://discourse.julialang.org/t/cuda-jl-mystery-vscode-julia-extension-works-fine-but-commandline-run-fails/84524/2 "2022-07-20T10:19:21Z")

</div>

I don’t have any issues when I run the code line by line in VSCode … I got the expected results.

---

<div class="post-metadata">

**Author:** ![Lian\_Yunlong](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lian_yunlong/32/14437_2.png) [@Lian\_Yunlong](https://discourse.julialang.org/u/Lian_Yunlong)\
**Post date:** [July 20, 2022, 10:40am UTC](https://discourse.julialang.org/t/cuda-jl-mystery-vscode-julia-extension-works-fine-but-commandline-run-fails/84524/3 "2022-07-20T10:40:41Z")

</div>

> [@Lian\_Yunlong](#):
>
> When I run the code from within the file (in VSCode + Julia extension), e

perhaps this is related to stream synchronization because when I profile the code, I found this line is involked many times and takes a lot of time:

> <https://github.com/JuliaGPU/CUDA.jl/blob/8e58e13df1e8e55ccf5e9b9fdc2a9200852fc845/lib/cudadrv/stream.jl#L169>

---

<div class="post-metadata">

**Author:** ![Lian\_Yunlong](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lian_yunlong/32/14437_2.png) [@Lian\_Yunlong](https://discourse.julialang.org/u/Lian_Yunlong)\
**Post date:** [July 20, 2022, 12:10pm UTC](https://discourse.julialang.org/t/cuda-jl-mystery-vscode-julia-extension-works-fine-but-commandline-run-fails/84524/4 "2022-07-20T12:10:17Z")

</div>

It is perhaps related to [stream-ordered allocations](https://juliagpu.org/post/2021-04-09-cuda_3.0/index.html#cuda_112_and_stream-ordered_allocations)

---

<div class="post-metadata">

**Author:** ![Lian\_Yunlong](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lian_yunlong/32/14437_2.png) [@Lian\_Yunlong](https://discourse.julialang.org/u/Lian_Yunlong)\
**Post date:** [July 20, 2022, 12:10pm UTC](https://discourse.julialang.org/t/cuda-jl-mystery-vscode-julia-extension-works-fine-but-commandline-run-fails/84524/5 "2022-07-20T12:10:33Z")

</div>

still working on this issue…  
I have pinned the error to the following:

inside a function `f(cu_A, cu_v)`, multiplication of the CuVector `cu_v` by the CuArray `cu_A` caused the error. `synchronize()` does not help. Running the code line by line in VSCode will not cause the error

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [July 21, 2022, 2:30pm UTC](https://discourse.julialang.org/t/cuda-jl-mystery-vscode-julia-extension-works-fine-but-commandline-run-fails/84524/6 "2022-07-21T14:30:01Z")

</div>