I expected the following to work:
d = Dict("a" => 1, "b" => 2)
@sync @distributed for (k, v) in d
But receive this error:
ERROR: On worker 2:
KeyError: key 1:1 not found
getindex at ./dict.jl:478 [inlined]
Iterating over a vector in parallel or iterating over the dict sequentially works fine.
Could someone explain what’s going on here?
Looking at what code
@macroexpand @distributed for (k, v) in d
gives a hint what’s going on, in particular the following line from the output:
for (k, v) = #25#R[#26#lo:#27#hi]
Looks like the
@distributed macro uses
length(d) to break the object into slices
lo:hi, in this case
2:2, but slicing like that doesn’t work with dicts. You can put the items in the dict into an array using
collect then it should work:
julia> @sync @distributed for (k, v) in collect(d)
From worker 2: b
From worker 3: a
Task (done) @0x00007f4b98df8010
I think arguably this should work without the
collect, so you could file an Issue, if there isn’t something similar already.
Thank you! that explains a lot.