DistributedArrays: basic example from doc fails

Specific Domains Julia at Scale
1

Hi all,
I am new to Julia and would like to try out DistributedArrays. However, the basic example from the Julia documentation fails for me. Trying to make sure that there is no issue with the installation, I completely removed Julia from the system, then reinstalled it, installed DistributedArrays and run it again - but it still does not work. Do I do something wrong, is the doc not up-to-date or is this a bug? See in the following 1) the shell output from my installation of Julia, followed by 2) the output of my installation of DistributedArrays, and then 3) the execution of the example from the doc (“(…)” represents repetitive omitted output):

1) My installation of Julia

omlins@dom101:~> ls ~/.julia
ls: cannot access '/users/omlins/.julia': No such file or directory
omlins@dom101:~> mkdir julia
omlins@dom101:~> cd julia/
omlins@dom101:~/julia> wget https://julialang-s3.julialang.org/bin/linux/x64/1.0/julia-1.0.2-linux-x86_64.tar.gz
--2018-12-10 18:41:06--  https://julialang-s3.julialang.org/bin/linux/x64/1.0/julia-1.0.2-linux-x86_64.tar.gz
Resolving julialang-s3.julialang.org (julialang-s3.julialang.org)... 151.101.54.49, 2a04:4e42:200::561, 2a04:4e42:600::561, ...
Connecting to julialang-s3.julialang.org (julialang-s3.julialang.org)|151.101.54.49|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 88860959 (85M) [application/octet-stream]
Saving to: ‘julia-1.0.2-linux-x86_64.tar.gz’

100%[=================================================>] 88'860'959  29.6MB/s   in 2.9s   

2018-12-10 18:41:15 (29.6 MB/s) - ‘julia-1.0.2-linux-x86_64.tar.gz’ saved [88860959/88860959]

omlins@dom101:~/julia> tar -xzf julia-1.0.2-linux-x86_64.tar.gz 
omlins@dom101:~/julia> cd julia-1.0.2/bin/
omlins@dom101:~/julia/julia-1.0.2/bin> ./julia 
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.0.2 (2018-11-08)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

2) My installation of DistributedArrays

julia> import Pkg; Pkg.status()
    Status `~/.julia/environments/v1.0/Project.toml`
  (empty environment)

julia> Pkg.installed()
Dict{String,Union{Nothing, VersionNumber}} with 0 entries

julia> Pkg.add("DistributedArrays")
   Cloning default registries into /users/omlins/.julia/registries
   Cloning registry General from "https://github.com/JuliaRegistries/General.git"
 Resolving package versions...
 Installed DistributedArrays ─ v0.5.1
 Installed Primes ──────────── v0.4.0
  Updating `~/.julia/environments/v1.0/Project.toml`
  [aaf54ef3] + DistributedArrays v0.5.1
  Updating `~/.julia/environments/v1.0/Manifest.toml`
  [aaf54ef3] + DistributedArrays v0.5.1
  [27ebfcd6] + Primes v0.4.0
  [2a0f44e3] + Base64 
  [8ba89e20] + Distributed 
  [b77e0a4c] + InteractiveUtils 
  [8f399da3] + Libdl 
  [37e2e46d] + LinearAlgebra 
  [56ddb016] + Logging 
  [d6f4376e] + Markdown 
  [9a3f8284] + Random 
  [9e88b42a] + Serialization 
  [6462fe0b] + Sockets 
  [2f01184e] + SparseArrays 
  [10745b16] + Statistics 
  [8dfed614] + Test 

julia>

3) The execution of the example from the doc

omlins@dom101:~/julia/julia-1.0.2/bin> ./julia -p 4
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.0.2 (2018-11-08)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> addprocs()
20-element Array{Int64,1}:
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25

julia> @everywhere using DistributedArrays
[ Info: Precompiling DistributedArrays [aaf54ef3-cdf8-58ed-94cc-d582ad619b94]
┌ Warning: The call to compilecache failed to create a usable precompiled cache file for DistributedArrays [aaf54ef3-cdf8-58ed-94cc-d582ad619b94]
│   exception = Required dependency Primes [27ebfcd6-29c5-5fa9-bf4b-fb8fc14df3ae] failed to load from a cache file.
└ @ Base loading.jl:969

(...)

┌ Warning: The call to compilecache failed to create a usable precompiled cache file for DistributedArrays [aaf54ef3-cdf8-58ed-94cc-d582ad619b94]
│   exception = Required dependency Primes [27ebfcd6-29c5-5fa9-bf4b-fb8fc14df3ae] failed to load from a cache file.
└ @ Base loading.jl:969
┌ Warning: The call to compilecache failed to create a usable precompiled cache file for DistributedArrays [aaf54ef3-cdf8-58ed-94cc-d582ad619b94]
│   exception = Invalid input in module list: expected Primes.
└ @ Base loading.jl:969

julia> B = ones(10_000) ./ 2;

julia> A = ones(10_000) .* π;

julia> C = 2 .* A ./ B;

julia> all(C .≈ 4*π)
true

julia> typeof(C)
Array{Float64,1}

julia> dB = distribute(B);

julia> dA = distribute(A);

julia> dC = 2 .* dA ./ dB;
ERROR: On worker 2:
MethodError: no method matching localtype(::Base.Broadcast.Broadcasted{Base.Broadcast.ArrayStyle{DArray},Nothing,typeof(*),Tuple{Int64,DArray{Float64,1,Array{Float64,1}}}})
Closest candidates are:
  localtype(::Type{DArray{T,N,S}}) where {T, N, S} at /users/omlins/.julia/packages/DistributedArrays/XV7NS/src/darray.jl:71
  localtype(::Type{SubArray{T,N,D,I,L} where L where I}) where {T, N, D} at /users/omlins/.julia/packages/DistributedArrays/XV7NS/src/darray.jl:72
  localtype(::Union{SubArray{T,N,D,I,L} where L where I where D<:DArray, DArray{T,N,A} where A} where N where T) at /users/omlins/.julia/packages/DistributedArrays/XV7NS/src/darray.jl:73
  ...
#119 at /users/omlins/.julia/packages/DistributedArrays/XV7NS/src/mapreduce.jl:32
map at ./tuple.jl:163
#118 at /users/omlins/.julia/packages/DistributedArrays/XV7NS/src/mapreduce.jl:28
#construct_localparts#3 at /users/omlins/.julia/packages/DistributedArrays/XV7NS/src/darray.jl:110
construct_localparts at /users/omlins/.julia/packages/DistributedArrays/XV7NS/src/darray.jl:110
#112 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Distributed/src/process_messages.jl:269
run_work_thunk at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Distributed/src/process_messages.jl:56
macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Distributed/src/process_messages.jl:269 [inlined]
#111 at ./task.jl:259
#remotecall_fetch#149(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::Function, ::Distributed.Worker, ::Function, ::Vararg{Any,N} where N) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Distributed/src/remotecall.jl:379
remotecall_fetch(::Function, ::Distributed.Worker, ::Function, ::Vararg{Any,N} where N) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Distributed/src/remotecall.jl:371
#remotecall_fetch#152(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::Function, ::Int64, ::Function, ::Vararg{Any,N} where N) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Distributed/src/remotecall.jl:406
remotecall_fetch(::Function, ::Int64, ::Function, ::Vararg{Any,N} where N) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Distributed/src/remotecall.jl:406
macro expansion at /users/omlins/.julia/packages/DistributedArrays/XV7NS/src/darray.jl:87 [inlined]
(::getfield(DistributedArrays, Symbol("##1#2")){Tuple{Int64,Int64},getfield(DistributedArrays, Symbol("##118#121")){Base.Broadcast.Broadcasted{Base.Broadcast.ArrayStyle{DArray},Tuple{Base.OneTo{Int64}},typeof(/),Tuple{Base.Broadcast.Broadcasted{Base.Broadcast.ArrayStyle{DArray},Nothing,typeof(*),Tuple{Int64,DArray{Float64,1,Array{Float64,1}}}},DArray{Float64,1,Array{Float64,1}}}},Tuple{Base.OneTo{Int64}}},Tuple{Int64},Array{Int64,1},Array{Tuple{UnitRange{Int64}},1},Array{Array{Int64,1},1},Channel{Any}})() at ./task.jl:259

...and 23 more exception(s).

Stacktrace:
 [1] sync_end(::Array{Any,1}) at ./task.jl:226
 [2] macro expansion at /users/omlins/.julia/packages/DistributedArrays/XV7NS/src/darray.jl:83 [inlined]
 [3] macro expansion at ./task.jl:244 [inlined]
 [4] DArray(::Tuple{Int64,Int64}, ::Function, ::Tuple{Int64}, ::Array{Int64,1}, ::Array{Tuple{UnitRange{Int64}},1}, ::Array{Array{Int64,1},1}) at /users/omlins/.julia/packages/DistributedArrays/XV7NS/src/darray.jl:82
 [5] DArray(::Function, ::Tuple{Int64}, ::Array{Int64,1}, ::Array{Int64,1}) at /users/omlins/.julia/packages/DistributedArrays/XV7NS/src/darray.jl:169
 [6] Type at /users/omlins/.julia/packages/DistributedArrays/XV7NS/src/darray.jl:176 [inlined]
 [7] DArray(::Function, ::Tuple{Int64}) at /users/omlins/.julia/packages/DistributedArrays/XV7NS/src/darray.jl:178
 [8] copy at /users/omlins/.julia/packages/DistributedArrays/XV7NS/src/mapreduce.jl:27 [inlined]
 [9] materialize(::Base.Broadcast.Broadcasted{Base.Broadcast.ArrayStyle{DArray},Nothing,typeof(/),Tuple{Base.Broadcast.Broadcasted{Base.Broadcast.ArrayStyle{DArray},Nothing,typeof(*),Tuple{Int64,DArray{Float64,1,Array{Float64,1}}}},DArray{Float64,1,Array{Float64,1}}}}) at ./broadcast.jl:748
 [10] top-level scope at none:0

julia>

The OS on the system is the following:

omlins@dom101:~/julia/julia-1.0.2/bin> cat /etc/os-release
NAME="SLES"
VERSION="12-SP3"
VERSION_ID="12.3"
PRETTY_NAME="SUSE Linux Enterprise Server 12 SP3"
ID="sles"
ANSI_COLOR="0;32"
CPE_NAME="cpe:/o:suse:sles:12:sp3"

I am completely stuck with this issue. Any help is highly appreciated!

Cheers,

Sam

2

try just “using DistributedArrays” without @everywhere prefix and see if it compiles successfully or not. if it gets compiled sucxcessfully then try “@everywhere …”.

3

Thanks for your comment @Ajaychat3. I did what you said. It compiled DistributedArrays without warning:

omlins@dom101:~/julia/julia-1.0.2/bin> ./julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.0.2 (2018-11-08)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> using DistributedArrays
[ Info: Recompiling stale cache file /users/omlins/.julia/compiled/v1.0/DistributedArrays/Gfdem.ji for DistributedArrays [aaf54ef3-cdf8-58ed-94cc-d582ad619b94]
julia>

Then, it failed however with the same error in the execution (see above in 3) The execution of the example from the doc).

4

I can reproduce the error with julia 1.0.2 on Ubuntu, using the latest tagged release of DistributedArrays (0.5.1). Howevever, it works with the master branch of DistributedArrays. From the pkg repl mode do

dev DistributedArrays

to checkout the master branch if you want to give it a try. Note that when you dev a package you’re cloning its git repository and you have to manually check for updates yourself using git. You can go back to the release version of the package by doing

free DistributedArrays

from the pkg repl.

5

Thank you very much @Pbellive ! After switching to the master branch of DistributedArrays as you adviced, the example run without error:

julia> @everywhere using DistributedArrays
[ Info: Recompiling stale cache file /users/omlins/.julia/compiled/v1.0/DistributedArrays/Gfdem.ji for DistributedArrays [aaf54ef3-cdf8-58ed-94cc-d582ad619b94]

julia> B = ones(10_000) ./ 2;

julia> A = ones(10_000) .* π;

julia> C = 2 .* A ./ B;

julia> all(C .≈ 4*π)
true

julia> typeof(C)
Array{Float64,1}

julia> dB = distribute(B);

julia> dA = distribute(A);

julia> dC = 2 .* dA ./ dB;

julia> all(dC .≈ 4*π)
true

julia> typeof(dC)
DArray{Float64,1,Array{Float64,1}}

This issue is therefore solved on the master of DistributedArrays.

Thanks again!

Cheers,

Sam

6

Just wanted to mention that, if you’re not planning to actually develop DistributedArrays, you can also do:

] add DistributedArrays#master
2 Likes