Dubious code: @spawn psort!(v, lo, mid)

New to Julia
1

Here is the code snippet taken from Announcing composable multi-threaded parallelism in Julia


import Base.Threads.@spawn

# sort the elements of `v` in place, from indices `lo` to `hi` inclusive
function psort!(v, lo::Int=1, hi::Int=length(v))

   # omitted above
    mid = (lo+hi)>>>1                 # find the midpoint

    half = @spawn psort!(v, lo, mid)  # task to sort the lower half; will run
    psort!(v, mid+1, hi)              # in parallel with the current call sorting
                                      # the upper half
    wait(half)                        # wait for the lower half to finish
    temp = v[lo:mid]

What’s dubious is this:
half = @spawn psort!(v, lo, mid)

When psort!(v) is spawned, the entire array that v points to must be copied to the remote process. so it is necessary to fetch(half) and copy the result to the local v[lo:mid].

Actually the above program never ends and keep spinning. More strangely, even if I put v[lo:mid] .= fetch(half), it never ends.

I think the following simple example confirms my understanding; data must be fetched from @spawn.

using Base.Threads, Distributed

arr = zeros(10)
@everywhere function update!(arr::AbstractArray)
    for i = 1:length(arr)
        arr[i] = i
    end
end

r = @spawn update!(arr)
wait(r)
println(arr)

the result of which is all-zero array without fetch.

What mistake am I making? or the example is really flawed?

2

Threads.@spawn doesn’t use multiple processes (distributed memory), it uses shared memory and there is no need to copy any data between tasks spawned in the same process.

3 Likes
3

Ah, I see. Indeed there are two @spawn: Threads.@spawn and Distributed.@spawn. The latter is deprecated. Thank you for the clarification :slight_smile: (It’s till strange why the program never terminated on my PC)