Distributed computing, not found package error

New to Julia
1

Hello, I need some help. I am using Julia 1.3.1 on Ubuntu 18.4. My code doesn’t run on a worker. What am I doing wrong?

I created a Julia module AppliSales.jl: (AppliMaster) pkg> add https://github.com/rbontekoe/AppliSales.jl.

Next code runs well
julia> using Distributed

julia> @everywhere using AppliSales

julia> addprocs(4)

jjulia> AppliSales.process()
3-element Array{AppliSales.Order,1}:
 AppliSales.Order("17052596140033858593", AppliSales.Organization("14577970130459819545", "Scrooge Investment Bank", "1180 Seven Seas Dr", "FL 32830", "Lake Buena Vista", "USA"), AppliSales.Training("LS", 2019-08-30T00:00:00, 2, "Learn Smiling", 1000.0), "PO-456", "Scrooge McDuck", "scrooge@duckcity.com", ["Scrooge McDuck"])
 AppliSales.Order("10457127966801272728", AppliSales.Organization("1196631451138768503", "Duck City Chronicals", "1185 Seven Seas Dr", "FL 32830", "Lake Buena Vista", "USA"), AppliSales.Training("LS", 2019-08-30T00:00:00, 2, "Learn Smiling", 1000.0), "DD-001", "Mickey Mouse", "mickey@duckcity.com", ["Mini Mouse", "Goofy"])  
 AppliSales.Order("5908989182124673668", AppliSales.Organization("15936294520763292015", "Donalds Hardware Store", "1190 Seven Seas Dr", "FL 32830", "Lake Buena Vista", "USA"), AppliSales.Training("LS", 2019-08-30T00:00:00, 2, "Learn Smiling", 1000.0), "", "Donald Duck", "donald@duckcity.com", ["Daisy Duck"])
However, when running the method on a worker, I get an error.

It tells me that it can not find the AppliSales package. Am I using @everywhere in the wrong way?

julia> @fetch AppliSales.process()
ERROR: On worker 2:
KeyError: key AppliSales [a1ddd20a-2e39-11ea-38f9-6b919ef027c3] not found
getindex at ./dict.jl:477 [inlined]
root_module at ./loading.jl:962 [inlined]
deserialize_module at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Serialization/src/Serialization.jl:894
handle_deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Serialization/src/Serialization.jl:799
deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Serialization/src/Serialization.jl:722
handle_deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Serialization/src/Serialization.jl:807
deserialize_fillarray! at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Serialization/src/Serialization.jl:1094
deserialize_array at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Serialization/src/Serialization.jl:1086
handle_deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Serialization/src/Serialization.jl:773
deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Serialization/src/Serialization.jl:722 [inlined]
deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Serialization/src/Serialization.jl:992
handle_deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Serialization/src/Serialization.jl:860
deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Serialization/src/Serialization.jl:722 [inlined]
deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Serialization/src/Serialization.jl:929
handle_deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Serialization/src/Serialization.jl:860
deserialize_fillarray! at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Serialization/src/Serialization.jl:1094
deserialize_array at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Serialization/src/Serialization.jl:1086
handle_deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Serialization/src/Serialization.jl:773
deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Serialization/src/Serialization.jl:722 [inlined]
deserialize_typename at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Serialization/src/Serialization.jl:1159
deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Distributed/src/clusterserialize.jl:68
handle_deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Serialization/src/Serialization.jl:860
deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Serialization/src/Serialization.jl:722
handle_deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Serialization/src/Serialization.jl:779
deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Serialization/src/Serialization.jl:722
handle_deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Serialization/src/Serialization.jl:782
deserialize at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Serialization/src/Serialization.jl:722 [inlined]
deserialize_msg at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Distributed/src/messages.jl:99
#invokelatest#1 at ./essentials.jl:709 [inlined]
invokelatest at ./essentials.jl:708 [inlined]
message_handler_loop at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Distributed/src/process_messages.jl:185
process_tcp_streams at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Distributed/src/process_messages.jl:142
#101 at ./task.jl:333
Stacktrace:
 [1] #remotecall_fetch#145 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Distributed/src/remotecall.jl:390 [inlined]
 [2] remotecall_fetch(::Function, ::Distributed.Worker) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Distributed/src/remotecall.jl:382
 [3] #remotecall_fetch#148(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(remotecall_fetch), ::Function, ::Int64) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Distributed/src/remotecall.jl:417
 [4] remotecall_fetch(::Function, ::Int64) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Distributed/src/remotecall.jl:417
 [5] top-level scope at none:0
2

addprocs()
should be called before other @everywhere using ...

The workers must be known before using the @everywhere macro.

2 Likes
3

@oheil, thank you for your suggestion. I tried it but get the same error.

In the past, I used Julia v1.1.0 and SQLite v0.8.x. I use a local channel with a dispatcher. It routes the data to the right task, which runs a Julia function remote. The application ran smoothly. The problem started when I switched to SQLite.jl v1.0.1. First, I thought that SQLite was causing the problem. However, the sample code uses only the AppliSales module, which doesn’t use SQLite.

Julia v1.3.1 could cause the problem. I will try it with an older version.

(v1.3) pkg> activate .
Activating environment at `~/julia-projects/tc/AppliMaster/Project.toml`

(AppliMaster) pkg> status
    Status `~/julia-projects/tc/AppliMaster/Project.toml`
  [153ef306] AppliGeneralLedger v0.1.0 #master (https://github.com/rbontekoe/AppliGeneralLedger.jl)
  [3941c6da] AppliInvoicing v0.2.1 #master (https://github.com/rbontekoe/AppliInvoicing.jl)
  [9e99b7bb] AppliSQLite v0.1.0 #master (https://github.com/rbontekoe/AppliSQLite.jl)
  [a1ddd20a] AppliSales v0.1.0 #master (https://github.com/rbontekoe/AppliSales.jl)
  [336ed68f] CSV v0.5.24
  [a10d1c49] DBInterface v2.0.0
  [a93c6f00] DataFrames v0.20.2
  [0aa819cd] SQLite v1.0.1
  [ade2ca70] Dates 

julia> using Distributed

julia> addprocs(4);

julia> @everywhere using AppliSales
ERROR: On worker 2:
ArgumentError: Package AppliSales [a1ddd20a-2e39-11ea-38f9-6b919ef027c3] is required but does not seem to be installed:
 - Run `Pkg.instantiate()` to install all recorded dependencies.

_require at ./loading.jl:993
require at ./loading.jl:922
#1 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Distributed/src/Distributed.jl:78
#105 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Distributed/src/process_messages.jl:290
run_work_thunk at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Distributed/src/process_messages.jl:79
run_work_thunk at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Distributed/src/process_messages.jl:88
#98 at ./task.jl:333

...and 3 more exception(s).

Stacktrace:
 [1] sync_end(::Array{Any,1}) at ./task.jl:300
 [2] _require_callback(::Base.PkgId) at ./task.jl:319
 [3] #invokelatest#1 at ./essentials.jl:709 [inlined]
 [4] invokelatest at ./essentials.jl:708 [inlined]
 [5] require(::Base.PkgId) at ./loading.jl:925
 [6] require(::Module, ::Symbol) at ./loading.jl:917
 [7] top-level scope at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.3/Distributed/src/macros.jl:199
4

Try

addprocs(n_procs; exeflags=`--project=$(Base.active_project())`)

instead – the currently active environment doesn’t propagate to processes started by addprocs by default.

5 Likes
5

@pfitzseb, thank you for your quick response.

Your solution, in combination with @oheil’s suggestion, solved the problem.

6

@pfitzseb is this still the most appropriate solution in Julia v1.4? Shouldn’t we file an issue about this exeflags options not defaulting to the most sensible value you suggested?

I spent some great amount of time debugging my script. Not a very user friendly experience this error on worker processes.

7

@juliohm, I received your suggestion by e-mail. Not sure @pfitzseb got it too (click on his username to send a message)

8

Nothing has changed about that, as far as I know.