`@everywhere using` fails for stdlib functions in distribution binaries

Internals & Design
1

Is this a bug?

image
image1898×604 154 KB

julia> using Distributed

julia> addprocs(4)
4-element Array{Int64,1}:
 2
 3
 4
 5

julia> using Distributed

julia> @everywhere using DistributedArrays
ERROR: ArgumentError: Package DistributedArrays not found in current path:
- Run `import Pkg; Pkg.add("DistributedArrays")` to install the DistributedArrays package.

Stacktrace:
 [1] require(::Module, ::Symbol) at ./loading.jl:885
 [2] top-level scope at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.4/Distributed/src/macros.jl:199

I do not have DistributedArrays installed. And if I want to use it @everywhere, the Stacktrace[2] will report a place:

/Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.4/Distributed/src/macros.jl:199

, which is nowhere on my computer. Should it be

/Applications/Julia-1.4.app/Contents/Resources/julia/share/julia/stdlib/v1.4/Distributed/src/macros.jl

It looks similar to another issue, but that was fixed.

julia> versioninfo()
Julia Version 1.4.0-DEV.435
Commit 3b11bf3add (2019-11-06 20:42 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin18.6.0)
  CPU: Intel(R) Xeon(R) W-2140B CPU @ 3.20GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)
Environment:
  JULIA_NUM_THREADS = 16
  JULIA_EDITOR = code
2 
(v1.2) pkg> add DistributedArrays
  Updating registry at `C:\Users\jerry\.julia\registries\General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
 Resolving package versions...
 Installed TimerOutputs ────── v0.5.1
 Installed AdvancedHMC ─────── v0.2.13
 Installed Primes ──────────── v0.4.0
 Installed FFTW ────────────── v1.1.0
....

julia> using Distributed

julia> addprocs(4)
4-element Array{Int64,1}:
 2
 3
 4
 5

julia> @everywhere using DistributedArrays
[ Info: Precompiling DistributedArrays [aaf54ef3-cdf8-58ed-94cc-d582ad619b94]

julia> versioninfo()
Julia Version 1.2.0
Commit c6da87ff4b (2019-08-20 00:03 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: AMD Ryzen 9 3900X 12-Core Processor
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, znver1)
Environment:
  JULIA_NUM_THREADS = 10
3

Sorry, I did not make it clear. My point is not I cannot @everywhere using DistributedArrays. It is if I cannot run it, the stacktrace returns

/Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.4/Distributed/src/macros.jl:199

but not

/Applications/Julia-1.4.app/Contents/Resources/julia/share/julia/stdlib/v1.4/Distributed/src/macros.jl

I was wondering why the path to macro.jl is not correct. Is it a similar issute to this?

4

but this line gave you the error, no? the errors are from distributed workers, because you didn’t add the package before adding procs (I think

5

Since the error is from v1.4/Distributed/src/macros.jl, I think it is a problem of my Julia v1.4 distribution. I am not sure what a buildbot/worker is. One thing I know is that before this issue was fixed, editing or lessing some code in standard libraries will also throw an error from a path /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.2/InteractiveUtils/src/editless.jl, which is not a real path on users’ computers. You can use Julia v1.2.0 to edit some code in a standard library to see this. So I am suggesting there might be a leftover bug that the path was not entirely fixed.

julia> edit(parse(IPv4, "127.0.0.1"))
ERROR: object is not callable
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] functionloc at ./reflection.jl:1150 [inlined]
 [3] edit(::IPv4) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.2/InteractiveUtils/src/editless.jl:99
 [4] top-level scope at REPL[3]:1

julia> versioninfo()
Julia Version 1.2.0
Commit c6da87ff4b (2019-08-20 00:03 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, broadwell)
Environment:
  JULIA_NUM_THREADS = 12