I am trying to parallelize a for-loop that updates array and scalar counters. I am able to reduce the scalar counters, but I get this error when I add the array counter. ERROR: LoadError: syntax: invalid syntax in "local" declaration around C:\Users\User\.julia\packages\FLoops\AN2BM\src\reduce.jl:449
This is the function I was using. It takes an input file and uses the data of each row to find the appropriate element of the array counter to increment.
function parallel_test(Input, N_rows)
counter_evens = 0
counter_odds = zeros(Int16, (N_rows))
@floop for i = 1:N_rows
a = round(Input[i,1]) # some processing based on input data
if a%2 == 0
c_evens = 1
@reduce(counter_evens += c_evens)
else
c_odds= 1
ind = round(rand()*a) # find array element to increment
@reduce(counter_odds[ind] += c_odds)
end
end
return counter_evens, counter_odds
end
Input_file = rand(1:9, (10,5)) .* rand()
evens, odds = parallel_test(Input_file , 10)
Am I using @reduce incorrectly, or is this configuration not possible with FLoops.jl?
counter_odds[ind] += ... is the problematic part (though the error message can be improved). In particular, you can’t use shared mutable state like this in parallel programming as it’d invoke a data race. Here is one way to do it
function parallel_test(Input, N_rows)
counter_evens = 0
@floop for i = 1:N_rows
a = round(Input[i,1])
if a%2 == 0
c_evens = 1
@reduce(counter_evens += c_evens)
else
c_odds= 1
ind = min(max(round(Int, rand()*a), 1), N_rows) # get an integer
another = ind => c_odds
@reduce() do (counter_odds = zeros(Int16, (N_rows)); another)
if another isa Pair
counter_odds[first(another)] += last(another)
else
counter_odds .+= another
end
end
end
end
return counter_evens, counter_odds
end
another isa Pair is a bit ugly “hack.” If you have a (hypothetical) OneHotVector, you can also write it as
another = OneHotVector(ind => c_odds) # s.t. another[ind] == c_odds and 0 elsewhere
@reduce() do (counter_odds = zeros(Int16, (N_rows)); another)
counter_odds .+= another
end
which clarifies the symmetry of this reduction. See also the histogram section in A quick introduction to data parallelism in Julia for a similar example. (I’m also writing another tutorial specifically for reduction with mutable states but it’s not finished yet.)