# CUDA CPU allocations with range

**URL:** <https://discourse.julialang.org/t/cuda-cpu-allocations-with-range/74549>\
**Category:** General Usage\
**Tags:** cuda\
**Created:** [January 13, 2022, 12:57pm UTC](https://discourse.julialang.org/t/cuda-cpu-allocations-with-range/74549 "2022-01-13T12:57:03Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![roflmaostc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/roflmaostc/32/30123_2.png) [@roflmaostc](https://discourse.julialang.org/u/roflmaostc)\
**Post date:** [January 13, 2022, 12:57pm UTC](https://discourse.julialang.org/t/cuda-cpu-allocations-with-range/74549/1 "2022-01-13T12:57:03Z")

</div>

Hi!

I was wondering, why `CuArray` of a range allocates the result on the CPU and moves it to the GPU afterwards? If I do the same directly with a broadcast call, it works flawless.  
Any suggestion how to prevent that?

```julia
julia> using CUDA

julia> xc = CUDA.rand(10_000); 

julia> CUDA.@time xc .= 1:10_000; # already compiled
  0.000097 seconds (7 CPU allocations: 480 bytes) 

julia> CUDA.@time CuArray(1:10_000); # already compiled, why CPU allocations?
  0.000051 seconds (8 CPU allocations: 78.344 KiB) (1 GPU allocation: 78.125 KiB, 22.71% memmgmt time) 

julia> CUDA.@time Array(1:10_000); 
  0.000010 seconds (2 CPU allocations: 78.172 KiB)

```

Best,

Felix

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [January 13, 2022, 1:03pm UTC](https://discourse.julialang.org/t/cuda-cpu-allocations-with-range/74549/2 "2022-01-13T13:03:05Z")

</div>

That CuArray constructor only accepts Array inputs, and converts whatever you pass it to to an Array. That’s more robust than trying to perform that operation on the GPU (the object might be mutable, for example). And constructing a CuArray isn’t supposed to be the expensive operation worth accelerating on the device; operations with the resulting object are.

---

<div class="post-metadata">

**Author:** ![roflmaostc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/roflmaostc/32/30123_2.png) [@roflmaostc](https://discourse.julialang.org/u/roflmaostc)\
**Post date:** [January 13, 2022, 1:04pm UTC](https://discourse.julialang.org/t/cuda-cpu-allocations-with-range/74549/3 "2022-01-13T13:04:50Z")

</div>

So the suggested way (e.g. a range) is to write `.=`?

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [January 13, 2022, 1:08pm UTC](https://discourse.julialang.org/t/cuda-cpu-allocations-with-range/74549/4 "2022-01-13T13:08:55Z")

</div>

Basically yes. That’s what such a constructor would do (and used to do at some point in the past). A custom kernel for ranges would probably outperform broadcast, but again this is unlikely to be a performance-critical operation.

---

<div class="post-metadata">

**Author:** ![roflmaostc](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/roflmaostc/32/30123_2.png) [@roflmaostc](https://discourse.julialang.org/u/roflmaostc)\
**Post date:** [January 13, 2022, 1:11pm UTC](https://discourse.julialang.org/t/cuda-cpu-allocations-with-range/74549/5 "2022-01-13T13:11:13Z")

</div>

Is there something like a convenient function?

```julia
function foo(AA::AbstractArray{T, N}) where {T, N}
    AAc = CUDA.Array{T, N}{undef, size(AA)...)
    AAc .= AA
    return AAc?
end
```

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [January 13, 2022, 1:15pm UTC](https://discourse.julialang.org/t/cuda-cpu-allocations-with-range/74549/6 "2022-01-13T13:15:16Z")

</div>

I don’t think 2 lines of code warrant a convenience function? If you care you can always create an additional outer constructor, a la [https://github.com/JuliaGPU/CUDA.jl/blob/e00ad245d96763ebac4cea2ce4a9c7d7b722bd58/src/array.jl#L286-L290](https://github.com/JuliaGPU/CUDA.jl/blob/e00ad245d96763ebac4cea2ce4a9c7d7b722bd58/src/array.jl#L286-L290). If this really has a significant impact you can create a PR on CUDA.jl 🙂