# Inplace array modification performances

**URL:** <https://discourse.julialang.org/t/inplace-array-modification-performances/84044>\
**Category:** GPU\
**Created:** [July 11, 2022, 10:17am UTC](https://discourse.julialang.org/t/inplace-array-modification-performances/84044 "2022-07-11T10:17:02Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![dgasteau](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dgasteau/32/37822_2.png) [@dgasteau](https://discourse.julialang.org/u/dgasteau)\
**Post date:** [July 11, 2022, 10:17am UTC](https://discourse.julialang.org/t/inplace-array-modification-performances/84044/1 "2022-07-11T10:17:02Z")

</div>

Hi,

I am trying to improve the memory management of some complex iterative procedure.  
Basically the procedure could be synthesized to  
for ii = 1:Nt  
do some partial arrays value update  
Compute some stuff (include fft ffti and other)  
end

I have some question about partial assignment of CuArrays and performances.  
I noticed that some code, which does not perform allocations in CPU, presents allocations on GPU.  
Researching on the forum has taught me that setindex! may not be the best with arguments of type view so I went for other approaches.  
The speed of computation is good in all the approaches, but considering that my actual set up implies iterative process on several large matrices, the memory management becomes a limiting factor.

How could I update an array with values of another array without making allocations?

```julia
using CUDA
using BenchmarkTools

Nx = 200
Ny = 200
Nz = 500

data = rand(Nx, Ny, Nz)
Cu_p = CUDA.zeros(Float32,Nx, Ny, Nz) # pressure
Cu_tr_data = CuArray(data) # pressure

t_index = 1

pos_x = 1 : Nx
pos_y = 1 : Ny
pos_z = 1

function func1(Cu_p, pos_x, pos_y, pos_z, Cu_tr_data, t_index)
    Cu_p[pos_x, pos_y, pos_z] .= view(Cu_tr_data, :, :, t_index)
end

function func2(Cu_p, pos_x, pos_y, pos_z, Cu_tr_data, t_index)
    copyto!(Cu_p[pos_x, pos_y, pos_z], view(Cu_tr_data, :, :, t_index))
end

source = view(Cu_p, pos_x, pos_y, pos_z)

function func3(source, Cu_tr_data, t_index)
    source .= view(Cu_tr_data, :, :, t_index)
end
function func4(source, Cu_tr_data, t_index)
    copyto!(source, view(Cu_tr_data, :, :, t_index))
end

@benchmark CUDA.@sync func1($Cu_p, $pos_x, $pos_y, $pos_z, $Cu_tr_data, $t_index)
@benchmark CUDA.@sync func2($Cu_p, $pos_x, $pos_y, $pos_z, $Cu_tr_data, $t_index)
@benchmark CUDA.@sync func3($source, $Cu_tr_data, $t_index)
@benchmark CUDA.@sync func4($source, $Cu_tr_data, $t_index)

```

 ![Screenshot from 2022-07-11 12-00-29](https://global.discourse-cdn.com/julialang/original/3X/c/3/c3c927deab8cf5b2dced38356d7f7f3e09ec563f.png)

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [July 11, 2022, 1:30pm UTC](https://discourse.julialang.org/t/inplace-array-modification-performances/84044/2 "2022-07-11T13:30:35Z")

</div>

Launching a kernel requires a allocations.

The allocations you report here are tiny, esp. compared to the total execution times. Are the allocations a problem, or why are you trying to avoid them?

---

<div class="post-metadata">

**Author:** ![dgasteau](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dgasteau/32/37822_2.png) [@dgasteau](https://discourse.julialang.org/u/dgasteau)\
**Post date:** [July 11, 2022, 1:46pm UTC](https://discourse.julialang.org/t/inplace-array-modification-performances/84044/3 "2022-07-11T13:46:55Z")

</div>

To make it short one iteration of my process, with the smaller array size I work with, takes about 0.07 sec per iteration, but I get a peak in execution time every 3 iteration step or so when gc is called. Ideally I would work with arrays 4 to 8 times larger.  
Given there is several hundred of iteration steps, I feel there is somewhere to gain time. I investigated towards minimizing overall allocations to better handle my memory and reduce gc calls.  
Every computation step is made inplace, all CuArrays are created at the beginning of the program, though I still have allocations.

---

<div class="post-metadata">

**Author:** ![maleadt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/maleadt/32/10097_2.png) [@maleadt](https://discourse.julialang.org/u/maleadt)\
**Post date:** [July 12, 2022, 6:35am UTC](https://discourse.julialang.org/t/inplace-array-modification-performances/84044/4 "2022-07-12T06:35:33Z")

</div>

> [@dgasteau](#):
>
> Every computation step is made inplace, all CuArrays are created at the beginning of the program, though I still have allocations.

You are looking at _CPU_ allocations. Use `CUDA.@time` to report GPU allocations.

---

<div class="post-metadata">

**Author:** ![dgasteau](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dgasteau/32/37822_2.png) [@dgasteau](https://discourse.julialang.org/u/dgasteau)\
**Post date:** [July 20, 2022, 7:09am UTC](https://discourse.julialang.org/t/inplace-array-modification-performances/84044/5 "2022-07-20T07:09:48Z")

</div>

Sorry for the late response, I could not have access to my GPU earlier.  
Indeed, I misunderstood the output of the @benchmark. Switching to CUDA.@time shows indeed no GPU allocations (except the copyto!(xx, view()) form, but I saw an other post on the forum talking about that).  
Thanks again.