What is the difference between in scope and loaded

General Usage
1

In the docs of parallel computing (https://docs.julialang.org/en/stable/manual/parallel-computing/#Code-Availability-and-Loading-Packages-1), there’s a sentence

using DummyModule causes the module to be loaded on all processes; however, the module is brought into scope only on the one executing the statement.

I do not understand this sentence. What is the difference between in scope and loaded?

2

I think this is meant:

julia> using Compat                                                                                                                                                      
                                                                                                                                                                         
julia> @compat 1                                                                                                                                                         
1                                                                                                                                                                        

julia> Compat.@compat 1                                                                                                                                                  
1                                                                                                                                                                        

julia> @everywhere Compat.@compat 1                                                                                                                                      
                                                                                                                                                                         
julia> @everywhere @compat 1                                                                                                                                             
ERROR: On worker 2:                                                                                                                                                      
UndefVarError: @compat not defined
eval at ./boot.jl:235
eval_ew_expr at ./distributed/macros.jl:116
#106 at ./distributed/process_messages.jl:268 [inlined]
run_work_thunk at ./distributed/process_messages.jl:56
macro expansion at ./distributed/process_messages.jl:268 [inlined]
#105 at ./event.jl:73
#remotecall_fetch#141(::Array{Any,1}, ::Function, ::Function, ::Base.Distributed.Worker, ::Expr, ::Vararg{Expr,N} where N) at ./distributed/remotecall.jl:354
remotecall_fetch(::Function, ::Base.Distributed.Worker, ::Expr, ::Vararg{Expr,N} where N) at ./distributed/remotecall.jl:346
#remotecall_fetch#144(::Array{Any,1}, ::Function, ::Function, ::Int64, ::Expr, ::Vararg{Expr,N} where N) at ./distributed/remotecall.jl:367
remotecall_fetch(::Function, ::Int64, ::Expr, ::Vararg{Expr,N} where N) at ./distributed/remotecall.jl:367
(::##7#9)() at ./distributed/macros.jl:102

...and 1 more exception(s).
                                                                                                                                                                         
Stacktrace:                                                                                                                                                              
 [1] sync_end() at ./task.jl:287                                                                                                                                         
 [2] macro expansion at ./distributed/macros.jl:112 [inlined]                                                                                                            
 [3] anonymous at ./<missing>:?

i.e. using Compat also executes import Compat on all nodes, but not using Compat. Whether this is good is disputed: https://github.com/JuliaLang/julia/issues/9245

1 Like
3

But this is working:

# Para.jl
module Para

export myidentity

function myidentity(x)
    return x
end

end

# mytest.jl
push!(LOAD_PATH, "D:\\Working\\temp") # This is my path

using Para

# I expect an UndefVarError here,
# using @everywhere does raise an UndefVarError here though.
println(pmap(myidentity, ['a', 'b', 'c']))

Executing julia -p 3 mytest.jl in the commandline correctly yields ['a', 'b', 'c'].

4

Yep, this is a bit odd. In particular the docs says:

help?> pmap
search: pmap TypeMapLevel TypeMapEntry promote_shape repmat typemax SparseMatrixCSC polygamma AbstractSparseMatrix PermutedDimsArray

  pmap([::AbstractWorkerPool], f, c...; distributed=true, batch_size=1, on_error=nothing, retry_delays=[]), retry_check=nothing) -> collection                      
...       
  Note that f must be made available to all worker processes; see Code Availability and Loading Packages for details.
...