Assigning to an array that is on the GPU

Specific Domains Machine Learning
1

Suppose an array is on the gpu. In assigning new values to it, should we put the right hand side of the assignment on the gpu first before doing the assignment. I.e.,

  A = cu(rand(5,1))
  B = zeros(5,1)

is this ok

   A .= B

or this is needed

  A .= cu(B)
2

Is the goal to concatenate A and B or to actually assign it a new value? You are broadcasting the assignment operator which I’m not familiar with.

Have a look at these examples and maybe it will answer your question:

using CuArrays

A = cu(rand(5,1))
B = zeros(5,1)

julia> C = vcat(A,B)
10×1 CuArray{Float64,2,Nothing}:
 0.8463374376296997
 0.604820966720581
 0.7880327105522156
 0.4833880066871643
 0.39194774627685547
 0.0
 0.0
 0.0
 0.0
 0.0

It looks like concatenating the two returns a CuArray. This can be further verified like so:

julia> typeof(C)
CuArray{Float64,2,Nothing}

If you wanted to push each item in B to A you could do this (but note the Warning message it produces):

A = cu(vec(rand(5,1)))
B = zeros(5,1)

julia> push!(A, B...)
┌ Warning: Performing scalar operations on GPU arrays: This is very slow, consider disallowing these operations with `allowscalar(false)`
└ @ GPUArrays C:\Users\mthel\.julia\packages\GPUArrays\WZupy\src\host\indexing.jl:43
10-element CuArray{Float32,1,Nothing}:
 0.21138956
 0.0049796794
 0.7363381
 0.58986074
 0.8373987
 0.0
 0.0
 0.0
 0.0
 0.0

For an actual re-assignment:

A = cu(rand(5,1))
B = zeros(5,1)

julia> A = cu(B)
5×1 CuArray{Float32,2,Nothing}:
 0.0
 0.0
 0.0
 0.0
 0.0
3

The goal is to update the array with new data (that is available on the cpu).

The .= syntax is important.

I believe that A .= B`` causes the contents of A to be updated with those of B but the memory location of A does not change, whereas A = cu(B)``` causes A to point to cu(B), so is pointing to a different area of memory.

Of course I should just try it also, however sometimes in asking your expert answer reveals the more general principle