Hi all,
I’ve encountered an issue with using the @fetchfrom macro to communicate data between different workers in a parallel computing setup in Julia. I am working on a problem where I need to manage a localized arrays ψ for each block in a parallel computation. I need to exchange information between neighboring blocks to perform certain calculations.
To achieve this, I used the @fetchfrom macro to fetch data from a neighboring worker. Here’s a simplified version of the setup:
@everywhere workers() begin
# Define the field `ψ` on each worker
ψ = zeros(Nx + padd, Ny + padd) .+ myid() #or some other way to distinguish them
# Find neighbors
neighbor_id = find_neighbor(myid())
# Attempt to fetch data from a neighboring worker
ψ[Nx_loc+padd*2,:] .= @fetchfrom neighbor_id ψ[padd+1,:]
end
The issue arises when I try to use the @fetchfrom macro to fetch data from a neighboring worker. It seems that if the array ψ is defined on the worker that is fetching the data (say worker 2), the fetched data doesn’t reflect the values from the other worker (say worker 4). Instead, it returns the local data from worker 2 itself.
- If
ψis not defined on the worker fetching the data (worker 2), everything works fine, and the correct data from worker 4 is received. - If
ψis defined on the worker fetching the data, it ends up fetching its own local data rather than the data from the other worker.
This seems to be a scope or naming conflict issue, where the local definition of ψ on worker 2 somehow overrides or interferes with the @fetchfrom operation, leading it to return local data rather than data from the specified remote worker.
Has anyone else encountered this issue or a similar one? Is there a way to ensure that @fetchfrom fetches the correct data from the specified worker, even when the array has the same name on both the sending and receiving workers? Any insights or workarounds would be greatly appreciated!