Passing symbol values using `@everywhere`?

General Usage
,
1

This seems like a bug:

julia> using Distributed; addprocs()
julia> classifier = :logistic_l1
julia> @everywhere classifier = $classifier
ERROR: On worker 2:
UndefVarError: logistic_l1 not defined
top-level scope at none:0
eval at ./boot.jl:331
#101 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Distributed/src/process_messages.jl:290
run_work_thunk at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Distributed/src/process_messages.jl:79
run_work_thunk at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Distributed/src/process_messages.jl:88
#94 at ./task.jl:358

...and 16 more exception(s).

Stacktrace:
 [1] sync_end(::Array{Any,1}) at ./task.jl:316
 [2] macro expansion at ./task.jl:335 [inlined]
 [3] remotecall_eval(::Module, ::Array{Int64,1}, ::Expr) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Distributed/src/macros.jl:217
 [4] top-level scope at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/Distributed/src/macros.jl:201
 [5] eval(::Module, ::Any) at ./boot.jl:331
 [6] eval_user_input(::Any, ::REPL.REPLBackend) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.4/REPL/src/REPL.jl:86
 [7] run_backend(::REPL.REPLBackend) at /home/davidlittle/.julia/packages/Revise/tV8FE/src/Revise.jl:1165
 [8] top-level scope at none:0

My current workaround is to just convert to a symbol on the worker, like so.

julia> @everywhere classifier = Symbol($(string(classifier)))

Is this expected behavior?

2

Actually I like your workaround, but how about

julia> using Distributed; addprocs()

julia> classifier = :logistic_l1

julia> @everywhere classifier = $(QuoteNode(classifier))

julia> @everywhere println(typeof(classifier))
Symbol
      From worker 9:    Symbol
      From worker 3:    Symbol
      From worker 8:    Symbol
      From worker 7:    Symbol
      From worker 4:    Symbol
      From worker 10:   Symbol
      From worker 2:    Symbol
      From worker 14:   Symbol
      From worker 11:   Symbol
      From worker 5:    Symbol
      From worker 12:   Symbol
      From worker 6:    Symbol
      From worker 15:   Symbol
      From worker 13:   Symbol

Is this less workaround? I don’t know.

3

Seems like a matter of taste: my personal choice would be to avoid QuoteNode; I think the Symbol($(string(classifier)) construct is likely to be more transparent to more people.

1 Like
4

Yes, thats what I thought too. Your “workaround” is easy to understand, but uses some more CPU on the workers. QuoteNote should be no cost at the workers. So it depends on the real problem to solve.
Actually it is not quite clear to me why

julia> :(@everywhere classifier = $(QuoteNode(classifier)))
:(#= REPL[13]:1 =# @everywhere classifier = :logistic_l1)

but

julia> :(@everywhere classifier = $classifier)
:(#= REPL[16]:1 =# @everywhere classifier = logistic_l1)

I think to understand this we have to look into the code of interpolation (but I prefer to go to bed now. Good night :sleeping:).