# Multiple GPUs with CuStateVec in CUDA.jl

**URL:** <https://discourse.julialang.org/t/multiple-gpus-with-custatevec-in-cuda-jl/113499>\
**Category:** GPU\
**Created:** [April 25, 2024, 1:17pm UTC](https://discourse.julialang.org/t/multiple-gpus-with-custatevec-in-cuda-jl/113499 "2024-04-25T13:17:21Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![tipfom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tipfom/32/208771_2.png) [@tipfom](https://discourse.julialang.org/u/tipfom)\
**Post date:** [April 25, 2024, 1:17pm UTC](https://discourse.julialang.org/t/multiple-gpus-with-custatevec-in-cuda-jl/113499/1 "2024-04-25T13:17:21Z")

</div>

Hello,

I’m trying to use multiple GPUs for computations involving CuStateVec from the CUDA.jl implementation, specifically the function `swapIndexBitsMultiDevice!`.  
However, this function always throws the error

> ArgumentError: cannot take the CPU address of a CuArray{ComplexF32, 1, CUDA.Mem.DeviceBuffer}

The following minimal working example allows to reproduce the behavior on my local installation of CUDA 12.3 (with custatevec version 1.5.0) on A100 GPUs.

```julia
using CUDA, cuStateVec

psis = map(collect(devices())) do d
    device!(d)
    CuStateVec(ComplexF32, 5)
end

swapIndexBitsMultiDevice!(
    convert(Vector{CuStateVec}, psis), collect(devices()), 
    [1=>2], Int32[], Int32[], 
    CUSTATEVEC_DEVICE_NETWORK_TYPE_SWITCH)

```

Sadly, I do not have access to a different cluster to test if this is a problem occuring only on my installation?  
Is there anything wrong with the MWE, specifically, are the psis initialized correctly?  
There are no multi-GPU examples or test functions available to compare the MWE to.

I’m looking forward to any insights!

Thanks in advance and kind regards,  
Tim

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [April 25, 2024, 1:33pm UTC](https://discourse.julialang.org/t/multiple-gpus-with-custatevec-in-cuda-jl/113499/2 "2024-04-25T13:33:39Z")

</div>

Pinging @kslimes, who wrote these wrappers.