# What causes data movement when using remotecall\_fetch?

**URL:** <https://discourse.julialang.org/t/what-causes-data-movement-when-using-remotecall-fetch/42279>\
**Category:** General Usage\
**Tags:** distributed\
**Created:** [June 30, 2020, 2:18am UTC](https://discourse.julialang.org/t/what-causes-data-movement-when-using-remotecall-fetch/42279 "2020-06-30T02:18:56Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![xtt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xtt/32/11035_2.png) [@xtt](https://discourse.julialang.org/u/xtt)\
**Post date:** [June 30, 2020, 2:18am UTC](https://discourse.julialang.org/t/what-causes-data-movement-when-using-remotecall-fetch/42279/1 "2020-06-30T02:18:56Z")

</div>

Hi,

I read the section [Data Movement](https://docs.julialang.org/en/v1/manual/parallel-computing/#Data-Movement-1) and [Global variables](https://docs.julialang.org/en/v1/manual/parallel-computing/#Global-variables-1) to understand when data is being transferred between processes. Still, I need some help here.

I try to define a global data on the main process, send this data to remote processes via `remotecall_retch`, update data on the main process, and finally use `remotecall_fetch` to see if data is resent.

The example code and output on my machine:

Example 1: Reassign the entire array object causes data movement

```julia
using Distributed
A = [1,2,3] # [1, 2, 3]
remotecall_fetch(()->sum(A), 2) # 6
A = [4,5,6] # [4, 5, 6]
remotecall_fetch(()->sum(A), 2) # 15

```

This is the same as expected: data is captured and sent to worker 2.

Example 2: Resent also happens if the value of array rather than its pointer is changed:

```julia
using Distributed
A = [1,2,3] # [1, 2, 3]
remotecall_fetch(()->sum(A), 2) # 6
println(pointer_from_objref(A)) # Ptr{Nothing} @0x00007f9ef00de380
@. A += 5 # [6, 7, 8]
println(pointer_from_objref(A)) # Ptr{Nothing} @0x00007f9ef00de380
remotecall_fetch(()->sum(A), 2) # 21

```

According to the doc **“Globals are re-sent to a destination worker only in the context of a remote call, and then only if its value has changed. Also, the cluster does not synchronize global bindings across nodes.”** still makes sense if I interpret “value has changed” as “any changes including the value of array”.

Example 3: Change the value of a member of a user-defined object does not cause the resent of data:

```julia
using Distributed
@everywhere mutable struct X
    a
end
x = X([1,2,3]) # X([1, 2, 3])
remotecall_fetch(()->sum(x.a), 2) # 6
@. x.a += 5 # [6, 7, 8]
remotecall_fetch(()->sum(x.a), 2) # 6
sum(x.a) # 21

```

* * *

A clear statement of my problem : **what does it mean by “value has changed” in the doc that causes data resent?**

My julia version is 1.1.1

If this is a duplication please let me know too.

---

<div class="post-metadata">

**Author:** ![marius311](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/marius311/32/3953_2.png) [@marius311](https://discourse.julialang.org/u/marius311)\
**Post date:** [June 30, 2020, 3:19am UTC](https://discourse.julialang.org/t/what-causes-data-movement-when-using-remotecall-fetch/42279/2 "2020-06-30T03:19:50Z")

</div>

> [@xtt](#):
>
> A clear statement of my problem : **what does it mean by “value has changed” in the doc that causes data resent?**

The answer is that Julia uses `hash` to determine whether the value has changed, and the default hash doesn’t recurse into your object’s fields, it just uses [`objectid(x)`](https://github.com/JuliaLang/julia/blob/39c278b728deb04c3a32d70e3e35dcef7822c0c0/base/hashing.jl#L23). I think there’s some discussion of this in various threads, but I agree, its mildly surprising.

In any case, you can fix it by defining your own hash which does involve the contents, e.g. something like:

```julia
Base.hash(x::X, h::UInt) = hash(X, hash(x.a, h))

```

After that, your last example works as expected.

---

<div class="post-metadata">

**Author:** ![xtt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/xtt/32/11035_2.png) [@xtt](https://discourse.julialang.org/u/xtt)\
**Post date:** [June 30, 2020, 5:05am UTC](https://discourse.julialang.org/t/what-causes-data-movement-when-using-remotecall-fetch/42279/3 "2020-06-30T05:05:55Z")

</div>

Thank you. This is exactly what I am looking for.