Hi all,
I am new to Julia and would like to try out DistributedArrays. However, the basic example from the Julia documentation fails for me. Trying to make sure that there is no issue with the installation, I completely removed Julia from the system, then reinstalled it, installed DistributedArrays and run it again - but it still does not work. Do I do something wrong, is the doc not up-to-date or is this a bug? See in the following 1) the shell output from my installation of Julia, followed by 2) the output of my installation of DistributedArrays, and then 3) the execution of the example from the doc (“(…)” represents repetitive omitted output):
1) My installation of Julia
omlins@dom101:~> ls ~/.julia
ls: cannot access '/users/omlins/.julia': No such file or directory
omlins@dom101:~> mkdir julia
omlins@dom101:~> cd julia/
omlins@dom101:~/julia> wget https://julialang-s3.julialang.org/bin/linux/x64/1.0/julia-1.0.2-linux-x86_64.tar.gz
--2018-12-10 18:41:06-- https://julialang-s3.julialang.org/bin/linux/x64/1.0/julia-1.0.2-linux-x86_64.tar.gz
Resolving julialang-s3.julialang.org (julialang-s3.julialang.org)... 151.101.54.49, 2a04:4e42:200::561, 2a04:4e42:600::561, ...
Connecting to julialang-s3.julialang.org (julialang-s3.julialang.org)|151.101.54.49|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 88860959 (85M) [application/octet-stream]
Saving to: ‘julia-1.0.2-linux-x86_64.tar.gz’
100%[=================================================>] 88'860'959 29.6MB/s in 2.9s
2018-12-10 18:41:15 (29.6 MB/s) - ‘julia-1.0.2-linux-x86_64.tar.gz’ saved [88860959/88860959]
omlins@dom101:~/julia> tar -xzf julia-1.0.2-linux-x86_64.tar.gz
omlins@dom101:~/julia> cd julia-1.0.2/bin/
omlins@dom101:~/julia/julia-1.0.2/bin> ./julia
_
_ _ _(_)_ | Documentation: https://docs.julialang.org
(_) | (_) (_) |
_ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help.
| | | | | | |/ _` | |
| | |_| | | | (_| | | Version 1.0.2 (2018-11-08)
_/ |\__'_|_|_|\__'_| | Official https://julialang.org/ release
|__/ |
2) My installation of DistributedArrays
julia> import Pkg; Pkg.status()
Status `~/.julia/environments/v1.0/Project.toml`
(empty environment)
julia> Pkg.installed()
Dict{String,Union{Nothing, VersionNumber}} with 0 entries
julia> Pkg.add("DistributedArrays")
Cloning default registries into /users/omlins/.julia/registries
Cloning registry General from "https://github.com/JuliaRegistries/General.git"
Resolving package versions...
Installed DistributedArrays ─ v0.5.1
Installed Primes ──────────── v0.4.0
Updating `~/.julia/environments/v1.0/Project.toml`
[aaf54ef3] + DistributedArrays v0.5.1
Updating `~/.julia/environments/v1.0/Manifest.toml`
[aaf54ef3] + DistributedArrays v0.5.1
[27ebfcd6] + Primes v0.4.0
[2a0f44e3] + Base64
[8ba89e20] + Distributed
[b77e0a4c] + InteractiveUtils
[8f399da3] + Libdl
[37e2e46d] + LinearAlgebra
[56ddb016] + Logging
[d6f4376e] + Markdown
[9a3f8284] + Random
[9e88b42a] + Serialization
[6462fe0b] + Sockets
[2f01184e] + SparseArrays
[10745b16] + Statistics
[8dfed614] + Test
julia>
3) The execution of the example from the doc
omlins@dom101:~/julia/julia-1.0.2/bin> ./julia -p 4
_
_ _ _(_)_ | Documentation: https://docs.julialang.org
(_) | (_) (_) |
_ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help.
| | | | | | |/ _` | |
| | |_| | | | (_| | | Version 1.0.2 (2018-11-08)
_/ |\__'_|_|_|\__'_| | Official https://julialang.org/ release
|__/ |
julia> addprocs()
20-element Array{Int64,1}:
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
julia> @everywhere using DistributedArrays
[ Info: Precompiling DistributedArrays [aaf54ef3-cdf8-58ed-94cc-d582ad619b94]
┌ Warning: The call to compilecache failed to create a usable precompiled cache file for DistributedArrays [aaf54ef3-cdf8-58ed-94cc-d582ad619b94]
│ exception = Required dependency Primes [27ebfcd6-29c5-5fa9-bf4b-fb8fc14df3ae] failed to load from a cache file.
└ @ Base loading.jl:969
(...)
┌ Warning: The call to compilecache failed to create a usable precompiled cache file for DistributedArrays [aaf54ef3-cdf8-58ed-94cc-d582ad619b94]
│ exception = Required dependency Primes [27ebfcd6-29c5-5fa9-bf4b-fb8fc14df3ae] failed to load from a cache file.
└ @ Base loading.jl:969
┌ Warning: The call to compilecache failed to create a usable precompiled cache file for DistributedArrays [aaf54ef3-cdf8-58ed-94cc-d582ad619b94]
│ exception = Invalid input in module list: expected Primes.
└ @ Base loading.jl:969
julia> B = ones(10_000) ./ 2;
julia> A = ones(10_000) .* π;
julia> C = 2 .* A ./ B;
julia> all(C .≈ 4*π)
true
julia> typeof(C)
Array{Float64,1}
julia> dB = distribute(B);
julia> dA = distribute(A);
julia> dC = 2 .* dA ./ dB;
ERROR: On worker 2:
MethodError: no method matching localtype(::Base.Broadcast.Broadcasted{Base.Broadcast.ArrayStyle{DArray},Nothing,typeof(*),Tuple{Int64,DArray{Float64,1,Array{Float64,1}}}})
Closest candidates are:
localtype(::Type{DArray{T,N,S}}) where {T, N, S} at /users/omlins/.julia/packages/DistributedArrays/XV7NS/src/darray.jl:71
localtype(::Type{SubArray{T,N,D,I,L} where L where I}) where {T, N, D} at /users/omlins/.julia/packages/DistributedArrays/XV7NS/src/darray.jl:72
localtype(::Union{SubArray{T,N,D,I,L} where L where I where D<:DArray, DArray{T,N,A} where A} where N where T) at /users/omlins/.julia/packages/DistributedArrays/XV7NS/src/darray.jl:73
...
#119 at /users/omlins/.julia/packages/DistributedArrays/XV7NS/src/mapreduce.jl:32
map at ./tuple.jl:163
#118 at /users/omlins/.julia/packages/DistributedArrays/XV7NS/src/mapreduce.jl:28
#construct_localparts#3 at /users/omlins/.julia/packages/DistributedArrays/XV7NS/src/darray.jl:110
construct_localparts at /users/omlins/.julia/packages/DistributedArrays/XV7NS/src/darray.jl:110
#112 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Distributed/src/process_messages.jl:269
run_work_thunk at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Distributed/src/process_messages.jl:56
macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Distributed/src/process_messages.jl:269 [inlined]
#111 at ./task.jl:259
#remotecall_fetch#149(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::Function, ::Distributed.Worker, ::Function, ::Vararg{Any,N} where N) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Distributed/src/remotecall.jl:379
remotecall_fetch(::Function, ::Distributed.Worker, ::Function, ::Vararg{Any,N} where N) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Distributed/src/remotecall.jl:371
#remotecall_fetch#152(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::Function, ::Int64, ::Function, ::Vararg{Any,N} where N) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Distributed/src/remotecall.jl:406
remotecall_fetch(::Function, ::Int64, ::Function, ::Vararg{Any,N} where N) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Distributed/src/remotecall.jl:406
macro expansion at /users/omlins/.julia/packages/DistributedArrays/XV7NS/src/darray.jl:87 [inlined]
(::getfield(DistributedArrays, Symbol("##1#2")){Tuple{Int64,Int64},getfield(DistributedArrays, Symbol("##118#121")){Base.Broadcast.Broadcasted{Base.Broadcast.ArrayStyle{DArray},Tuple{Base.OneTo{Int64}},typeof(/),Tuple{Base.Broadcast.Broadcasted{Base.Broadcast.ArrayStyle{DArray},Nothing,typeof(*),Tuple{Int64,DArray{Float64,1,Array{Float64,1}}}},DArray{Float64,1,Array{Float64,1}}}},Tuple{Base.OneTo{Int64}}},Tuple{Int64},Array{Int64,1},Array{Tuple{UnitRange{Int64}},1},Array{Array{Int64,1},1},Channel{Any}})() at ./task.jl:259
...and 23 more exception(s).
Stacktrace:
[1] sync_end(::Array{Any,1}) at ./task.jl:226
[2] macro expansion at /users/omlins/.julia/packages/DistributedArrays/XV7NS/src/darray.jl:83 [inlined]
[3] macro expansion at ./task.jl:244 [inlined]
[4] DArray(::Tuple{Int64,Int64}, ::Function, ::Tuple{Int64}, ::Array{Int64,1}, ::Array{Tuple{UnitRange{Int64}},1}, ::Array{Array{Int64,1},1}) at /users/omlins/.julia/packages/DistributedArrays/XV7NS/src/darray.jl:82
[5] DArray(::Function, ::Tuple{Int64}, ::Array{Int64,1}, ::Array{Int64,1}) at /users/omlins/.julia/packages/DistributedArrays/XV7NS/src/darray.jl:169
[6] Type at /users/omlins/.julia/packages/DistributedArrays/XV7NS/src/darray.jl:176 [inlined]
[7] DArray(::Function, ::Tuple{Int64}) at /users/omlins/.julia/packages/DistributedArrays/XV7NS/src/darray.jl:178
[8] copy at /users/omlins/.julia/packages/DistributedArrays/XV7NS/src/mapreduce.jl:27 [inlined]
[9] materialize(::Base.Broadcast.Broadcasted{Base.Broadcast.ArrayStyle{DArray},Nothing,typeof(/),Tuple{Base.Broadcast.Broadcasted{Base.Broadcast.ArrayStyle{DArray},Nothing,typeof(*),Tuple{Int64,DArray{Float64,1,Array{Float64,1}}}},DArray{Float64,1,Array{Float64,1}}}}) at ./broadcast.jl:748
[10] top-level scope at none:0
julia>
The OS on the system is the following:
omlins@dom101:~/julia/julia-1.0.2/bin> cat /etc/os-release
NAME="SLES"
VERSION="12-SP3"
VERSION_ID="12.3"
PRETTY_NAME="SUSE Linux Enterprise Server 12 SP3"
ID="sles"
ANSI_COLOR="0;32"
CPE_NAME="cpe:/o:suse:sles:12:sp3"
I am completely stuck with this issue. Any help is highly appreciated!
Cheers,
Sam