fetch to communicate large arrays between nodes on a compute cluster. This results in more memory allocation than I would like. Is there an in-place version of
fetch that uses pre-allocated memory? Here is an example to illustrate:
addprocs(2) function main() f1 = @spawnat workers() rand(1000) f2 = @spawnat workers() rand(1000) x = zeros(1000) x[:] = fetch(f1) # I would like to do something like fetch!(f1,x) y = sum(x) x[:] = fetch(f2) z = sum(x) @show y,z end main()
If I understand correctly, both calls to
fetch will allocate memory for an array that is the same size as
x. Subsequently this memory is copied from the
x. This extra allocation and copy is what I want to try and avoid.
Thanks for taking time to read this, and for any ideas towards a solution?