# Passing CuArray to PyTorch Tensor

**URL:** <https://discourse.julialang.org/t/passing-cuarray-to-pytorch-tensor/23067>\
**Category:** GPU\
**Created:** [April 12, 2019, 8:57am UTC](https://discourse.julialang.org/t/passing-cuarray-to-pytorch-tensor/23067 "2019-04-12T08:57:17Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![gdestouet](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdestouet/32/6175_2.png) [@gdestouet](https://discourse.julialang.org/u/gdestouet)\
**Post date:** [April 12, 2019, 8:57am UTC](https://discourse.julialang.org/t/passing-cuarray-to-pytorch-tensor/23067/1 "2019-04-12T08:57:17Z")

</div>

Hello everyone,

I am actually using this to pass CuArrays to PyTorch Tensor :

```julia
using PyCall
tc = pyimport("torch")
using CuArrays 
x=cu(randn(1024,1024))
@time tc.from_numpy(Array(x)).cuda()
 0.025017 seconds (62 allocations: 20.00 MiB)
 ...

```

Since `Array(x)` is actually moving x from GPU to CPU, and `.cuda()` from CPU to GPU, I was wondering if there is a way to stay on the GPU and convert CuArrays to PyTorch tensor without having to go back and forth between CPU and GPU.

Directly applying `tc.from_numpy()` on CuArrays is not working and gives a SegFault

Let me know if you have any ideas about how to do this.

Thank you !

---

<div class="post-metadata">

**Author:** ![yatra9](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yatra9/32/3697_2.png) [@yatra9](https://discourse.julialang.org/u/yatra9)\
**Post date:** [April 12, 2019, 3:30pm UTC](https://discourse.julialang.org/t/passing-cuarray-to-pytorch-tensor/23067/2 "2019-04-12T15:30:23Z")

</div>

One solution is to create an PyTorch tensor, convert it to a CuArray by `Base.unsafe_wrap`, and then manipulate it using CuArrays.jl.  
No copy at all.

```julia
x_tc = tc.empty((1024,1024), dtype=tc.float64).cuda()
x = Base.unsafe_wrap(CuArray, CuPtr{Float64}(x_tc.data_ptr()), (1024, 1024))
randn!(x)
# randn!(x) modifies PyTorch tensor x_tc because PyTorch tensor x_tc and CuArray x share the same memory.

```

I haven’t actually tried this code, so there might be minor mistakes…

---

<div class="post-metadata">

**Author:** ![yatra9](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yatra9/32/3697_2.png) [@yatra9](https://discourse.julialang.org/u/yatra9)\
**Post date:** [April 12, 2019, 3:42pm UTC](https://discourse.julialang.org/t/passing-cuarray-to-pytorch-tensor/23067/3 "2019-04-12T15:42:31Z")

</div>

Or, though I don’t know if that’s really possible, possibly CuArray can be converted directly to PyTorch tensor through [`torch.utils.dlpack.from_dlpack`](https://pytorch.org/docs/stable/dlpack.html?highlight=dlpack).

> **[GitHub - dmlc/dlpack: common in-memory tensor structure](https://github.com/dmlc/dlpack)**
>
> common in-memory tensor structure . Contribute to dmlc/dlpack development by creating an account on GitHub.

---

<div class="post-metadata">

**Author:** ![gdestouet](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdestouet/32/6175_2.png) [@gdestouet](https://discourse.julialang.org/u/gdestouet)\
**Post date:** [April 25, 2019, 4:14pm UTC](https://discourse.julialang.org/t/passing-cuarray-to-pytorch-tensor/23067/4 "2019-04-25T16:14:37Z")

</div>

Thank you @yatra9 for your answer !  
The following code you proposed, is working with CuArrays v1.0.2 with CUDAdrv v3.0.0 and NNlib v0.6.0

> [@yatra9](#):
>
> ```julia
> x_tc = tc.empty((1024,1024), dtype=tc.float64).cuda()
> x = Base.unsafe_wrap(CuArray, CuPtr{Float64}(x_tc.data_ptr()), (1024, 1024))
> randn!(x)
> # randn!(x) modifies PyTorch tensor x_tc because PyTorch tensor x_tc and CuArray x share the same memory.
> 
> ```

---

<div class="post-metadata">

**Author:** ![JosePereiraUA](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/josepereiraua/32/37640_2.png) [@JosePereiraUA](https://discourse.julialang.org/u/JosePereiraUA)\
**Post date:** [October 13, 2021, 9:49am UTC](https://discourse.julialang.org/t/passing-cuarray-to-pytorch-tensor/23067/5 "2021-10-13T09:49:37Z")

</div>

Hi everyone. I’ve attempted to use the pointer trick, with no success. I wish to transfer the following Tensor to a CuArray:

```julia
julia> t
PyObject tensor([[-0.0478, 0.1304, 0.0551],
        [0.1353, -0.1581, 0.0776],
        [-0.0804, 0.0388, -0.0387],
        [-0.0254, -0.0076, -0.0433],
        [0.0183, -0.0035, -0.0508]], device='cuda:0')

```

I used:

```julia
julia> x = Base.unsafe_wrap(CuArray, CuPtr{Float64}(t.data_ptr()), t[:shape])
5×3 CuArray{Float64, 2, CUDA.Mem.DeviceBuffer}:
  6.26441e-10 -6.16828e-14 1.51954e-19
  7.74017e-10 -1.62869e-22 6.22065e-24
  7.03467e-12 -2.82217e-19 8.2082e-29
  2.75222e-14 -1.10739e-18 1.41569e-28
 -9.14529e-16 1.54366e-20 2.38145e-24

```

And, as you can see, the resulting array is not the same data. I assume something is wrong with the CuPtr pointer?

Thank you,

---

<div class="post-metadata">

**Author:** ![albheim](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/albheim/32/34660_2.png) [@albheim](https://discourse.julialang.org/u/albheim)\
**Post date:** [October 13, 2021, 11:11am UTC](https://discourse.julialang.org/t/passing-cuarray-to-pytorch-tensor/23067/6 "2021-10-13T11:11:06Z")

</div>

First thought was if you really had 64 bit floats in the tensor? Seems pretty common to run things with 32 bits on the GPU so could be an easy miss.

But I don’t really have any clue about how the GPU things work, so that is my best guess.

---

<div class="post-metadata">

**Author:** ![JosePereiraUA](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/josepereiraua/32/37640_2.png) [@JosePereiraUA](https://discourse.julialang.org/u/JosePereiraUA)\
**Post date:** [October 13, 2021, 11:17am UTC](https://discourse.julialang.org/t/passing-cuarray-to-pytorch-tensor/23067/7 "2021-10-13T11:17:26Z")

</div>

It’s true, I’ve tried with Float32 as well, same result. The original question also used Float64, I think.

---

<div class="post-metadata">

**Author:** ![PabloZubieta](https://avatars.discourse-cdn.com/v4/letter/p/ee7513/32.png) [@PabloZubieta](https://discourse.julialang.org/u/PabloZubieta)\
**Post date:** [February 22, 2022, 12:01am UTC](https://discourse.julialang.org/t/passing-cuarray-to-pytorch-tensor/23067/8 "2022-02-22T00:01:37Z")

</div>

There’s now [[ANN] DLPack.jl - Share CPU and CUDA arrays between Julia and Python](https://discourse.julialang.org/t/ann-announcing-dlpack-jl/76871/2) in case anyone here finds useful.