# DistributedArrays: basic example from doc fails

**URL:** <https://discourse.julialang.org/t/distributedarrays-basic-example-from-doc-fails/18551>\
**Category:** Julia at Scale\
**Tags:** question\
**Created:** [December 11, 2018, 9:14am UTC](https://discourse.julialang.org/t/distributedarrays-basic-example-from-doc-fails/18551 "2018-12-11T09:14:56Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![samo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/samo/32/35398_2.png) [@samo](https://discourse.julialang.org/u/samo)\
**Post date:** [December 11, 2018, 9:14am UTC](https://discourse.julialang.org/t/distributedarrays-basic-example-from-doc-fails/18551/1 "2018-12-11T09:14:57Z")

</div>

Hi all,  
I am new to Julia and would like to try out **DistributedArrays**. However, [the basic example from the Julia documentation](https://docs.julialang.org/en/v1.0/manual/parallel-computing/#Noteworthy-external-packages-1) fails for me. Trying to make sure that there is no issue with the installation, I completely removed Julia from the system, then reinstalled it, installed DistributedArrays and run it again - but it still does not work. **Do I do something wrong, is the doc not up-to-date or is this a bug?** See in the following 1) the shell output from my installation of Julia, followed by 2) the output of my installation of DistributedArrays, and then 3) the execution of the example from the doc (“(…)” represents repetitive omitted output):

## 1) My installation of Julia

```bash
omlins@dom101:~> ls ~/.julia
ls: cannot access '/users/omlins/.julia': No such file or directory
omlins@dom101:~> mkdir julia
omlins@dom101:~> cd julia/
omlins@dom101:~/julia> wget https://julialang-s3.julialang.org/bin/linux/x64/1.0/julia-1.0.2-linux-x86_64.tar.gz
--2018-12-10 18:41:06-- https://julialang-s3.julialang.org/bin/linux/x64/1.0/julia-1.0.2-linux-x86_64.tar.gz
Resolving julialang-s3.julialang.org (julialang-s3.julialang.org)... 151.101.54.49, 2a04:4e42:200::561, 2a04:4e42:600::561, ...
Connecting to julialang-s3.julialang.org (julialang-s3.julialang.org)|151.101.54.49|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 88860959 (85M) [application/octet-stream]
Saving to: ‘julia-1.0.2-linux-x86_64.tar.gz’

100%[=================================================>] 88'860'959 29.6MB/s in 2.9s   

2018-12-10 18:41:15 (29.6 MB/s) - ‘julia-1.0.2-linux-x86_64.tar.gz’ saved [88860959/88860959]

omlins@dom101:~/julia> tar -xzf julia-1.0.2-linux-x86_64.tar.gz 
omlins@dom101:~/julia> cd julia-1.0.2/bin/
omlins@dom101:~/julia/julia-1.0.2/bin> ./julia 
               _
   _ _ _(_)_ | Documentation: https://docs.julialang.org
  (_) | (_) (_) |
   _ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` | |
  | | |_| | | | (_| | | Version 1.0.2 (2018-11-08)
 _/ |\ __'_|_|_|\__'_| | Official https://julialang.org/ release
|__/ |

```

## 2) My installation of DistributedArrays

```julia
julia> import Pkg; Pkg.status()
    Status `~/.julia/environments/v1.0/Project.toml`
  (empty environment)

julia> Pkg.installed()
Dict{String,Union{Nothing, VersionNumber}} with 0 entries

julia> Pkg.add("DistributedArrays")
   Cloning default registries into /users/omlins/.julia/registries
   Cloning registry General from "https://github.com/JuliaRegistries/General.git"
 Resolving package versions...
 Installed DistributedArrays ─ v0.5.1
 Installed Primes ──────────── v0.4.0
  Updating `~/.julia/environments/v1.0/Project.toml`
  [aaf54ef3] + DistributedArrays v0.5.1
  Updating `~/.julia/environments/v1.0/Manifest.toml`
  [aaf54ef3] + DistributedArrays v0.5.1
  [27ebfcd6] + Primes v0.4.0
  [2a0f44e3] + Base64 
  [8ba89e20] + Distributed 
  [b77e0a4c] + InteractiveUtils 
  [8f399da3] + Libdl 
  [37e2e46d] + LinearAlgebra 
  [56ddb016] + Logging 
  [d6f4376e] + Markdown 
  [9a3f8284] + Random 
  [9e88b42a] + Serialization 
  [6462fe0b] + Sockets 
  [2f01184e] + SparseArrays 
  [10745b16] + Statistics 
  [8dfed614] + Test 

julia> 

```

## 3) The execution of the example from the doc

```julia
omlins@dom101:~/julia/julia-1.0.2/bin> ./julia -p 4
               _
   _ _ _(_)_ | Documentation: https://docs.julialang.org
  (_) | (_) (_) |
   _ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` | |
  | | |_| | | | (_| | | Version 1.0.2 (2018-11-08)
 _/ |\ __'_|_|_|\__'_| | Official https://julialang.org/ release
|__/ |

```

```julia
julia> addprocs()
20-element Array{Int64,1}:
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25

julia> @everywhere using DistributedArrays
[Info: Precompiling DistributedArrays [aaf54ef3-cdf8-58ed-94cc-d582ad619b94]
┌ Warning: The call to compilecache failed to create a usable precompiled cache file for DistributedArrays [aaf54ef3-cdf8-58ed-94cc-d582ad619b94]
│ exception = Required dependency Primes [27ebfcd6-29c5-5fa9-bf4b-fb8fc14df3ae] failed to load from a cache file.
└ @ Base loading.jl:969

(...)

┌ Warning: The call to compilecache failed to create a usable precompiled cache file for DistributedArrays [aaf54ef3-cdf8-58ed-94cc-d582ad619b94]
│ exception = Required dependency Primes [27ebfcd6-29c5-5fa9-bf4b-fb8fc14df3ae] failed to load from a cache file.
└ @ Base loading.jl:969
┌ Warning: The call to compilecache failed to create a usable precompiled cache file for DistributedArrays [aaf54ef3-cdf8-58ed-94cc-d582ad619b94]
│ exception = Invalid input in module list: expected Primes.
└ @ Base loading.jl:969

julia> B = ones(10_000) ./ 2;

julia> A = ones(10_000) .* π;

julia> C = 2 .* A ./ B;

julia> all(C .≈ 4*π)
true

julia> typeof(C)
Array{Float64,1}

julia> dB = distribute(B);

julia> dA = distribute(A);

julia> dC = 2 .* dA ./ dB;
ERROR: On worker 2:
MethodError: no method matching localtype(::Base.Broadcast.Broadcasted{Base.Broadcast.ArrayStyle{DArray},Nothing,typeof(*),Tuple{Int64,DArray{Float64,1,Array{Float64,1}}}})
Closest candidates are:
  localtype(::Type{DArray{T,N,S}}) where {T, N, S} at /users/omlins/.julia/packages/DistributedArrays/XV7NS/src/darray.jl:71
  localtype(::Type{SubArray{T,N,D,I,L} where L where I}) where {T, N, D} at /users/omlins/.julia/packages/DistributedArrays/XV7NS/src/darray.jl:72
  localtype(::Union{SubArray{T,N,D,I,L} where L where I where D<:DArray, DArray{T,N,A} where A} where N where T) at /users/omlins/.julia/packages/DistributedArrays/XV7NS/src/darray.jl:73
  ...
#119 at /users/omlins/.julia/packages/DistributedArrays/XV7NS/src/mapreduce.jl:32
map at ./tuple.jl:163
#118 at /users/omlins/.julia/packages/DistributedArrays/XV7NS/src/mapreduce.jl:28
#construct_localparts#3 at /users/omlins/.julia/packages/DistributedArrays/XV7NS/src/darray.jl:110
construct_localparts at /users/omlins/.julia/packages/DistributedArrays/XV7NS/src/darray.jl:110
#112 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Distributed/src/process_messages.jl:269
run_work_thunk at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Distributed/src/process_messages.jl:56
macro expansion at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Distributed/src/process_messages.jl:269 [inlined]
#111 at ./task.jl:259
#remotecall_fetch#149(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::Function, ::Distributed.Worker, ::Function, ::Vararg{Any,N} where N) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Distributed/src/remotecall.jl:379
remotecall_fetch(::Function, ::Distributed.Worker, ::Function, ::Vararg{Any,N} where N) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Distributed/src/remotecall.jl:371
#remotecall_fetch#152(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::Function, ::Int64, ::Function, ::Vararg{Any,N} where N) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Distributed/src/remotecall.jl:406
remotecall_fetch(::Function, ::Int64, ::Function, ::Vararg{Any,N} where N) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Distributed/src/remotecall.jl:406
macro expansion at /users/omlins/.julia/packages/DistributedArrays/XV7NS/src/darray.jl:87 [inlined]
(::getfield(DistributedArrays, Symbol("##1#2")){Tuple{Int64,Int64},getfield(DistributedArrays, Symbol("##118#121")){Base.Broadcast.Broadcasted{Base.Broadcast.ArrayStyle{DArray},Tuple{Base.OneTo{Int64}},typeof(/),Tuple{Base.Broadcast.Broadcasted{Base.Broadcast.ArrayStyle{DArray},Nothing,typeof(*),Tuple{Int64,DArray{Float64,1,Array{Float64,1}}}},DArray{Float64,1,Array{Float64,1}}}},Tuple{Base.OneTo{Int64}}},Tuple{Int64},Array{Int64,1},Array{Tuple{UnitRange{Int64}},1},Array{Array{Int64,1},1},Channel{Any}})() at ./task.jl:259

...and 23 more exception(s).

Stacktrace:
 [1] sync_end(::Array{Any,1}) at ./task.jl:226
 [2] macro expansion at /users/omlins/.julia/packages/DistributedArrays/XV7NS/src/darray.jl:83 [inlined]
 [3] macro expansion at ./task.jl:244 [inlined]
 [4] DArray(::Tuple{Int64,Int64}, ::Function, ::Tuple{Int64}, ::Array{Int64,1}, ::Array{Tuple{UnitRange{Int64}},1}, ::Array{Array{Int64,1},1}) at /users/omlins/.julia/packages/DistributedArrays/XV7NS/src/darray.jl:82
 [5] DArray(::Function, ::Tuple{Int64}, ::Array{Int64,1}, ::Array{Int64,1}) at /users/omlins/.julia/packages/DistributedArrays/XV7NS/src/darray.jl:169
 [6] Type at /users/omlins/.julia/packages/DistributedArrays/XV7NS/src/darray.jl:176 [inlined]
 [7] DArray(::Function, ::Tuple{Int64}) at /users/omlins/.julia/packages/DistributedArrays/XV7NS/src/darray.jl:178
 [8] copy at /users/omlins/.julia/packages/DistributedArrays/XV7NS/src/mapreduce.jl:27 [inlined]
 [9] materialize(::Base.Broadcast.Broadcasted{Base.Broadcast.ArrayStyle{DArray},Nothing,typeof(/),Tuple{Base.Broadcast.Broadcasted{Base.Broadcast.ArrayStyle{DArray},Nothing,typeof(*),Tuple{Int64,DArray{Float64,1,Array{Float64,1}}}},DArray{Float64,1,Array{Float64,1}}}}) at ./broadcast.jl:748
 [10] top-level scope at none:0

julia> 

```

The OS on the system is the following:

```bash
omlins@dom101:~/julia/julia-1.0.2/bin> cat /etc/os-release
NAME="SLES"
VERSION="12-SP3"
VERSION_ID="12.3"
PRETTY_NAME="SUSE Linux Enterprise Server 12 SP3"
ID="sles"
ANSI_COLOR="0;32"
CPE_NAME="cpe:/o:suse:sles:12:sp3"

```

I am completely stuck with this issue. **Any help is highly appreciated!**

Cheers,

Sam

---

<div class="post-metadata">

**Author:** ![Ajaychat3](https://avatars.discourse-cdn.com/v4/letter/a/ecd19e/32.png) [@Ajaychat3](https://discourse.julialang.org/u/Ajaychat3)\
**Post date:** [December 11, 2018, 10:07am UTC](https://discourse.julialang.org/t/distributedarrays-basic-example-from-doc-fails/18551/2 "2018-12-11T10:07:31Z")

</div>

try just “using DistributedArrays” without @everywhere prefix and see if it compiles successfully or not. if it gets compiled sucxcessfully then try “@everywhere …”.

---

<div class="post-metadata">

**Author:** ![samo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/samo/32/35398_2.png) [@samo](https://discourse.julialang.org/u/samo)\
**Post date:** [December 11, 2018, 1:03pm UTC](https://discourse.julialang.org/t/distributedarrays-basic-example-from-doc-fails/18551/3 "2018-12-11T13:03:16Z")

</div>

Thanks for your comment @Ajaychat3. I did what you said. It compiled _DistributedArrays_ without warning:

```julia
omlins@dom101:~/julia/julia-1.0.2/bin> ./julia
               _
   _ _ _(_)_ | Documentation: https://docs.julialang.org
  (_) | (_) (_) |
   _ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` | |
  | | |_| | | | (_| | | Version 1.0.2 (2018-11-08)
 _/ |\ __'_|_|_|\__'_| | Official https://julialang.org/ release
|__/ |

julia> using DistributedArrays
[Info: Recompiling stale cache file /users/omlins/.julia/compiled/v1.0/DistributedArrays/Gfdem.ji for DistributedArrays [aaf54ef3-cdf8-58ed-94cc-d582ad619b94]
julia>

```

Then, it failed however with the same error in the execution (see above in **3) The execution of the example from the doc**).

---

<div class="post-metadata">

**Author:** ![Pbellive](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pbellive/32/3604_2.png) [@Pbellive](https://discourse.julialang.org/u/Pbellive)\
**Post date:** [December 11, 2018, 5:07pm UTC](https://discourse.julialang.org/t/distributedarrays-basic-example-from-doc-fails/18551/4 "2018-12-11T17:07:35Z")

</div>

I can reproduce the error with julia 1.0.2 on Ubuntu, using the latest tagged release of DistributedArrays (0.5.1). Howevever, it works with the master branch of DistributedArrays. From the pkg repl mode do

```julia
dev DistributedArrays

```

to checkout the master branch if you want to give it a try. Note that when you dev a package you’re cloning its git repository and you have to manually check for updates yourself using git. You can go back to the release version of the package by doing

```julia
free DistributedArrays

```

from the pkg repl.

---

<div class="post-metadata">

**Author:** ![samo](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/samo/32/35398_2.png) [@samo](https://discourse.julialang.org/u/samo)\
**Post date:** [December 11, 2018, 5:32pm UTC](https://discourse.julialang.org/t/distributedarrays-basic-example-from-doc-fails/18551/5 "2018-12-11T17:32:43Z")

</div>

Thank you very much @Pbellive ! After switching to the master branch of _DistributedArrays_ as you adviced, the example run without error:

```julia
julia> @everywhere using DistributedArrays
[Info: Recompiling stale cache file /users/omlins/.julia/compiled/v1.0/DistributedArrays/Gfdem.ji for DistributedArrays [aaf54ef3-cdf8-58ed-94cc-d582ad619b94]

julia> B = ones(10_000) ./ 2;

julia> A = ones(10_000) .* π;

julia> C = 2 .* A ./ B;

julia> all(C .≈ 4*π)
true

julia> typeof(C)
Array{Float64,1}

julia> dB = distribute(B);

julia> dA = distribute(A);

julia> dC = 2 .* dA ./ dB;

julia> all(dC .≈ 4*π)
true

julia> typeof(dC)
DArray{Float64,1,Array{Float64,1}}

```

This issue is therefore solved on the master of _DistributedArrays_.

Thanks again!

Cheers,

Sam

---

<div class="post-metadata">

**Author:** ![kevbonham](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kevbonham/32/216165_2.png) [@kevbonham](https://discourse.julialang.org/u/kevbonham)\
**Post date:** [December 11, 2018, 7:46pm UTC](https://discourse.julialang.org/t/distributedarrays-basic-example-from-doc-fails/18551/6 "2018-12-11T19:46:23Z")

</div>

> [@Pbellive](#):
>
> From the pkg repl mode do
> 
> ```julia
> dev DistributedArrays
> 
> ```
> 
> to checkout the master branch if you want to give it a try

Just wanted to mention that, if you’re not planning to actually develop `DistributedArrays`, you can also do:

```julia
] add DistributedArrays#master

```