I am wondering how to parallelize an expensive computation but where I
specify which workers do the work?
By way of background, in my use case each worker does a large computation on worker-specific objects (i.e. stored in Main on each worker - I realise this is clunky, but other considerations have dictated this approach) and returns a vector. I want then take a sum over this vector of vectors from each worker. If i didn’t care which workers did the tasks, summing over the output of pmap would be perfect. However, I need to be sure each worker does exactly one task.
It seems like i should be able to do this with a @distributed for loop and @spawnat, but I am not making progress. In particular, I am not sure how to move the output vector resulting from the @spawnat from the worker to the master process .
Any help greatly appreciated!
Update: Solved by this post