How to best use DistributedArray.jl

General Usage Performance
, ,
1

I have a couple of questions about Distributed arrays.

  • I have to simulate a lattice, and for every point of this lattice I have to save a collection of matrices. This is because I’m interested in the links connecting each points and not the points themselves. For example, a 3D lattice of length 5 in each dimension, would have 125 points, and each of this point would have 3 matrices, for a total of 375 matrices. Is it efficient to represent this as a 3D Distributed array, where each cell contains a static, mutable vector (from StaticArrays.jl) of length 3 containing matrices? The reason I do this is to ensure that when the lattice gets split up between processes by DistributedArrays.jl, I am 100% sure that a process that owns a point, also owns every link of this point. This wouldn’t necessarily true if I directly use a 4D DArray, where one dimension would represent the list of matrices at each point.
  • I have written a simple function that retrieves an element of the lattice. To do this, I simply access the element I’m interested in using the standard index notation. Now, should I split the function in two cases, where one case uses the local version of the lattice (using localpart()) and the other uses the entire DArray? Or does DistributedArray.jl already take care of this and I can simply access DArrays using index notation? I’m asking because most of the algorithm that each process will implement is going to be reading from the local part of the lattice, while sometimes needing to read from a cell in another process. My understanding is that it is not necessary to implemenent two methods since DistributedArrays.jl already takes care of this, but I want to be sure.

I hope I was clear enough.

2

It has been almost a week without a reply. It might help if you provide a MWE (maybe a serial version) of what you aim to do. The thing with distributed computations is that they are tricky to pull off. Even if your problem looks nice and parallelizable on paper, in reality overhead can cause performance to diminish.