# DistributedArrays: unexpected behavior before modifying localpart

**URL:** <https://discourse.julialang.org/t/distributedarrays-unexpected-behavior-before-modifying-localpart/7355>\
**Category:** Julia at Scale\
**Created:** [November 28, 2017, 11:07am UTC](https://discourse.julialang.org/t/distributedarrays-unexpected-behavior-before-modifying-localpart/7355 "2017-11-28T11:07:16Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![mikeT](https://avatars.discourse-cdn.com/v4/letter/m/9de053/32.png) [@mikeT](https://discourse.julialang.org/u/mikeT)\
**Post date:** [November 28, 2017, 11:07am UTC](https://discourse.julialang.org/t/distributedarrays-unexpected-behavior-before-modifying-localpart/7355/1 "2017-11-28T11:07:16Z")

</div>

I have been trying to learn enough about distributed arrays to solving a simple PDE in parallel; however, there appears to be an error associated with accessing localpart data right before modifying it. I’m trying to understand why and to see if I’m on the right path with DArrays.

Here is a working example:

```julia
using Base.Test
@everywhere using DistributedArrays
addprocs(4-nprocs())
dArray=distribute( collect(linspace(1,6,6)) )

@parallel for p in procs(dArray)
    firstLocalId=2*p-3
    localpart(dArray)[1:2,:]=1
    println("indexing at ",firstLocalId," gives ",dArray[firstLocalId,1])
end

@test sum(dArray)==6

```

It prints out the expected:  
From worker 3: indexing at 3 gives 1.0  
From worker 4: indexing at 5 gives 1.0  
From worker 2: indexing at 1 gives 1.0  
and the test passes.

However, if I move the print to before where I am modifying the data:

```julia
@parallel for p in procs(dArray)
    firstLocalId=2*p-3
    println("indexing at ",firstLocalId," gives ",dArray[firstLocalId,1])
    localpart(dArray)[1:2,:]=1
end

```

I still get a sensible output:  
From worker 3: indexing at 3 gives 3.0  
From worker 4: indexing at 5 gives 5.0  
From worker 2: indexing at 1 gives 1.0  
but the test fails (I think because there was some crash in the parallel executions somewhere?).

I also have two more related questions:

1. Why does Julia not crash when I have a syntax error or runtime error within an @parallel? It runs to completion but leaves the distributed array unchanged (which is why I added the @test to my example). Is this expected behaviour to have Julia not throw errors within parallel executions?
2. Is `@parallel for i in procs()` the same as

```julia
for p in procs()
    @spawnat p begin
       ...
    end
end

```

For my examples, the results seem similar. Is one better performing than the other? Thanks.

---

<div class="post-metadata">

**Author:** ![vchuravy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/vchuravy/32/8_2.png) [@vchuravy](https://discourse.julialang.org/u/vchuravy)\
**Post date:** [November 29, 2017, 4:33pm UTC](https://discourse.julialang.org/t/distributedarrays-unexpected-behavior-before-modifying-localpart/7355/2 "2017-11-29T16:33:33Z")

</div>

The documentation for `@parallel` states:

```julia
  Note that without a reducer function, @parallel executes asynchronously, i.e. it spawns independent tasks on all available workers and returns immediately without waiting for
  completion. To wait for completion, prefix the call with @sync, like :

  @sync @parallel for var = range
      body
  end

```

So in both cases you should write a `@sync` in fron of your `@parallel` loop wich will also catch and show any errors.

`@parallel` will be faster than your manual for loop since it will start the distributed computation async.  
It is equivalent to:

```julia
for p in procs()
    @async @spawnat p begin
       ...
    end
end

```