Understanding pid and RRID in 'Future' objects (related to @spawn and @spawnat in documentation)

moustachio-belvedere · March 2, 2018, 6:08pm

In the documentation on Parallel Computing the first code example contains:

julia> r = remotecall(rand, 2, 2, 2)
Future(2, 1, 4, Nullable{Any}())

julia> s = @spawnat 2 1 .+ fetch(r)
Future(2, 1, 5, Nullable{Any}())

Which makes sense to me (I think) because the first number field of the Future object, the pid, is 2 because both belong to process 2 and fetch is a no-op.

However, the second example:

julia> r = @spawn rand(2,2)
Future(2, 1, 4, Nullable{Any}())

julia> s = @spawn 1 .+ fetch(r)
Future(3, 1, 5, Nullable{Any}())

The pid for the object s appears to be 3, which I interpret as it belonging to process 3. Is this correct? If so, the text below this second example does not make sense:

In this case, @spawn is smart enough to perform the computation on the process that owns r, so the fetch() will be a no-op (no work is done).

findmyway · July 25, 2018, 4:12am

I have checked the source code. It seems the @spawn is not that smart yet…

github.com

JuliaLang/julia/blob/master/stdlib/Distributed/src/macros.jl

# This file is a part of Julia. License is MIT: https://julialang.org/license

let nextidx = Threads.Atomic{Int}(0)
    global nextproc
    function nextproc()
        idx = Threads.atomic_add!(nextidx, 1)
        return workers()[(idx % nworkers()) + 1]
    end
end

spawnat(p, thunk) = remotecall(thunk, p)

spawn_somewhere(thunk) = spawnat(nextproc(),thunk)

"""
    @spawn expr

Create a closure around an expression and run it on an automatically-chosen process,
returning a [`Future`](@ref) to the result.
This macro is deprecated; `@spawnat :any expr` should be used instead.

This file has been truncated. show original

Can anyone help clarify this ?

moustachio-belvedere · July 25, 2018, 10:09am

Unfortunately I can’t edit the question anymore but in hindsight this doesn’t seem like a ‘first steps’ question as the answer may be quite involved

moustachio-belvedere · July 25, 2018, 5:37pm

If there’s no response before, I will ask around at JuliaCon2018 regarding this and report back. If I can’t figure it out by then I’ll post an issue on Github.

moustachio-belvedere · July 31, 2018, 10:01am

Update, decided to go ahead and put an issue on Github:

https://github.com/JuliaLang/julia/issues/28350