I’m trying to use pmap form Distributed.jl for parallel computing in the runtests.jl file of a package (MyPkg) that is in development. The code is like:
# runtests.jl
using Distributed: @everywhere, addprocs, pmap
using MyPkg
myvect = [...] # The vector that is to be computed by `myfunc` parallelly.
addprocs(2)
@time result = pmap(myfunc, myvect); # `myfunc` is defined in `MyPkg.jl` (and other `.jl` files included by it)
If the last line is map(myfunc, myvect) the code can run normally. But when I change it to map(myfunc, myvect), it repots:
ERROR: On worker 2:
KeyError: key MyPkg[8f2d0ece-bb0f-4d0c-a9d4-8a8c66038d81] not found
Stacktrace:
[1] getindex
@ .\dict.jl:498 [inlined]
[2] macro expansion
@ .\lock.jl:267 [inlined]
[3] root_module
@ .\loading.jl:1878
[4] deserialize_module
@ D:\Software\julia-1.10.3\share\julia\stdlib\v1.10\Serialization\src\Serialization.jl:994
[5] handle_deserialize
@ D:\Software\julia-1.10.3\share\julia\stdlib\v1.10\Serialization\src\Serialization.jl:896
[6] deserialize
@ D:\Software\julia-1.10.3\share\julia\stdlib\v1.10\Serialization\src\Serialization.jl:814
[7] deserialize_datatype
@ D:\Software\julia-1.10.3\share\julia\stdlib\v1.10\Serialization\src\Serialization.jl:1398
[8] handle_deserialize
@ D:\Software\julia-1.10.3\share\julia\stdlib\v1.10\Serialization\src\Serialization.jl:867
[9] deserialize
@ D:\Software\julia-1.10.3\share\julia\stdlib\v1.10\Serialization\src\Serialization.jl:814
[10] handle_deserialize
@ D:\Software\julia-1.10.3\share\julia\stdlib\v1.10\Serialization\src\Serialization.jl:874
[11] deserialize
@ D:\Software\julia-1.10.3\share\julia\stdlib\v1.10\Serialization\src\Serialization.jl:814 [inlined]
[12] deserialize_global_from_main
@ D:\Software\julia-1.10.3\share\julia\stdlib\v1.10\Distributed\src\clusterserialize.jl:160
[13] #5
@ D:\Software\julia-1.10.3\share\julia\stdlib\v1.10\Distributed\src\clusterserialize.jl:72 [inlined]
[14] foreach
@ .\abstractarray.jl:3097
[15] deserialize
@ D:\Software\julia-1.10.3\share\julia\stdlib\v1.10\Distributed\src\clusterserialize.jl:72
[16] handle_deserialize
@ D:\Software\julia-1.10.3\share\julia\stdlib\v1.10\Serialization\src\Serialization.jl:960
[17] deserialize
@ D:\Software\julia-1.10.3\share\julia\stdlib\v1.10\Serialization\src\Serialization.jl:814
[18] handle_deserialize
@ D:\Software\julia-1.10.3\share\julia\stdlib\v1.10\Serialization\src\Serialization.jl:871
[19] deserialize
@ D:\Software\julia-1.10.3\share\julia\stdlib\v1.10\Serialization\src\Serialization.jl:814
[20] handle_deserialize
@ D:\Software\julia-1.10.3\share\julia\stdlib\v1.10\Serialization\src\Serialization.jl:874
[21] deserialize
@ D:\Software\julia-1.10.3\share\julia\stdlib\v1.10\Serialization\src\Serialization.jl:814 [inlined]
[22] deserialize_msg
@ D:\Software\julia-1.10.3\share\julia\stdlib\v1.10\Distributed\src\messages.jl:87
[23] #invokelatest#2
@ .\essentials.jl:892 [inlined]
[24] invokelatest
@ .\essentials.jl:889 [inlined]
[25] message_handler_loop
@ D:\Software\julia-1.10.3\share\julia\stdlib\v1.10\Distributed\src\process_messages.jl:176
[26] process_tcp_streams
@ D:\Software\julia-1.10.3\share\julia\stdlib\v1.10\Distributed\src\process_messages.jl:133
[27] #103
@ D:\Software\julia-1.10.3\share\julia\stdlib\v1.10\Distributed\src\process_messages.jl:121
Stacktrace:
[1] (::Base.var"#1033#1035")(x::Task)
@ Base .\asyncmap.jl:171
[2] foreach(f::Base.var"#1033#1035", itr::Vector{Any})
@ Base .\abstractarray.jl:3097
[3] maptwice(wrapped_f::Function, chnl::Channel{Any}, worker_tasks::Vector{Any}, c::Vector{Any})
@ Base .\asyncmap.jl:171
[4] wrap_n_exec_twice
@ .\asyncmap.jl:147 [inlined]
[5] #async_usemap#1018
@ .\asyncmap.jl:97 [inlined]
[6] async_usemap
@ .\asyncmap.jl:78 [inlined]
[7] #asyncmap#1017
@ .\asyncmap.jl:75 [inlined]
[8] asyncmap
@ .\asyncmap.jl:74 [inlined]
[9] pmap(f::Function, p::Distributed.WorkerPool, c::Vector{…}; distributed::Bool, batch_size::Int64, on_error::Nothing, retry_delays::Vector{…}, retry_check::Nothing)
@ Distributed D:\Software\julia-1.10.3\share\julia\stdlib\v1.10\Distributed\src\pmap.jl:126
[10] pmap(f::Function, p::Distributed.WorkerPool, c::Vector{Any})
@ Distributed D:\Software\julia-1.10.3\share\julia\stdlib\v1.10\Distributed\src\pmap.jl:99
[11] pmap(f::Function, c::Vector{Any}; kwargs::@Kwargs{})
@ Distributed D:\Software\julia-1.10.3\share\julia\stdlib\v1.10\Distributed\src\pmap.jl:156
[12] macro expansion
@ .\timing.jl:279 [inlined]
[13] top-level scope
@ d:\name\MyPkg\test\runtests.jl:35
