Remote process can not call function

General Usage
1

Hi,
this is my first try to setup remote processes in Julia and I am running into the following error:

julia> using Distributed

julia> @everywhere function worker(); println("Hi");end

julia> addprocs([("ip-address",1)])
1-element Array{Int64,1}:
 2

julia> fut = @spawnat 2 begin; sleep(0.5); println("Hi"); end
Future(2, 1, 3, nothing)

julia>       From worker 2:	Hi

julia> fut = @spawnat 2 worker()
Future(2, 1, 4, nothing)

julia> fetch(fut)
ERROR: On worker 2:
UndefVarError: #worker not defined
deserialize_datatype at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Serialization/src/Serialization.jl:1115
handle_deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Serialization/src/Serialization.jl:771
deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Serialization/src/Serialization.jl:731
handle_deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Serialization/src/Serialization.jl:778
deserialize_global_from_main at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Serialization/src/Serialization.jl:731
#5 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Distributed/src/clusterserialize.jl:72 [inlined]
foreach at ./abstractarray.jl:1866
deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Distributed/src/clusterserialize.jl:72
handle_deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Serialization/src/Serialization.jl:856
deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Serialization/src/Serialization.jl:731
handle_deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Serialization/src/Serialization.jl:775
deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Serialization/src/Serialization.jl:731
handle_deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Serialization/src/Serialization.jl:778
deserialize_msg at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Serialization/src/Serialization.jl:731
#invokelatest#1 at ./essentials.jl:742 [inlined]
invokelatest at ./essentials.jl:741 [inlined]
message_handler_loop at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Distributed/src/process_messages.jl:160
process_tcp_streams at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Distributed/src/process_messages.jl:117
#105 at ./task.jl:259
Stacktrace:
 [1] #remotecall_fetch#149(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::Function, ::Distributed.Worker, ::Distributed.RRID) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Distributed/src/remotecall.jl:379
 [2] remotecall_fetch(::Function, ::Distributed.Worker, ::Distributed.RRID, ::Vararg{Any,N} where N) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Distributed/src/remotecall.jl:371
 [3] #remotecall_fetch#152 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Distributed/src/remotecall.jl:406 [inlined]
 [4] remotecall_fetch at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Distributed/src/remotecall.jl:406 [inlined]
 [5] call_on_owner at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Distributed/src/remotecall.jl:479 [inlined]
 [6] fetch(::Future) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/Distributed/src/remotecall.jl:511
 [7] top-level scope at none:0

What could have gone wrong here? If you need any further information which can help find an explanation/solution, please tell me.

2

You need to add the workers first, then define your function.

3

Thanks for the quick answer! For some reason I thought it has to be the other way around… It is probably too late already :smiley:

4

@fredrikekre So, if I would put the code above in a script, the performance page for Julia suggest to put everything in e.g. a main-function. How do I define the worker functions in that case? In the main-function itself? What is the best practice in that case?

5

It does not suggest to put everything in the main function, you can have something like

function worker()
    # ...
end

function main()
    # ...
end
6

@fredrikekre I had something like the following in mind. Is this a good approach for a script?

using Distributed 
using OnlyLocallyNeededModules

addprocs(...)

@everywhere begin
    using GloballyNeededModules

    function worker_function()
          #...
    end
end

function main()
     # distributing work, fetching results, etc
end

main()
7

Sure.

8

@fredrikekre Ok. Thanks for all your help!