Confusing julia behavior. @everywhere macro changes the scope of local variables to global

General Usage
, ,
1

I just encountered a very confusing julia behavior. I always thought that variables defined inside a function remain local to that function. But in the following example, the scope changes.

I define a simple function as below

using Distributed
addprocs(2)

function f()
    @everywhere x = myid()
    @everywhere println("x = ", x)
end

Executing the following code

f()

gives the result

x = 1
From worker 2:    x = 2
From worker 3:    x = 3

But since x is defined inside the function, I would expect the variable x to be not defined outside the function. However, upon executing the following code

x

I get the result

1

Even more confusing is the execution of the following code

@fetchfrom 3 x

which again gives

1

This is super confusing behavior. First, how does x become available outside the function? Second, why are all the processors/cores returning the same value of x? This can create unintended memory leak issues. Thank you for your help.

2

The documentation contradicts your assumptions regarding scope:

Distributed.@everywhere ā€” Macro

@everywhere [procs()] expr

Execute an expression under Main on all procs .

2 Likes
3

What you want is probably

@everywhere function f()
    x = myid()
    println("x = ", x)
end

which makes the function available everywhere (the code gets evaluated in all Main modules). Followed by

@everywhere f()

which calls the function on every worker.

4

The @fetchfrom behaviour on a global variable on the other hand really baffles me, what is going on here?

using Distributed
addprocs(2)
@everywhere x = myid()
@everywhere println(x)
# 1
#       From worker 2:    2
#       From worker 3:    3
@fetchfrom 2 x
# returns 1
@everywhere println(x)
# 1
#       From worker 2:    1 # this is now changed! :o
#       From worker 3:    3
1 Like
5

More importantly, it uses eval, which works in global scope.

2 Likes
6

I find this a bit confusing as well. One thing that often adds clarity is to replace macros with the code executed. In this case:

julia> using MacroTools

julia> prettify(@macroexpand(@fetchfrom 2 x))
:(Distributed.remotecall_fetch((()->x), 2))

so @fetchfrom 2 x actually means

remotecall_fetch(() -> x, 2)

i.e. it creates a closure around x, and then executes the result on pid 2. I can see how that ends up returning 1 given the closure is created on pid 1 and therefore likely closes over the local x, but Iā€™m not sure why it changes the value of x on pid 2.

The way to get the value of x from other processes is to directly ask for the symbol defined in Main on those processes, as described in this SO answer:

julia> @fetchfrom 3 getfield(Main, :x)
3

julia> @everywhere println(x)
1
      From worker 2:	2
      From worker 3:	3
2 Likes