mapreduce(f, op, X; dims = dim) can be used, but it seems that the parallel analog
pmapreduce from the package
ParallelUtilities does not support the
How would one use simple parallel functions to perform the same operation and still get a big speed up?
X is of dims
(very very big, big, very big) (say
(10^5, 10^2, 10^3)) and the reduction happens on the first (largest) dimension.
f are about
op are typically
Example of what I am doing
N = 10^5 n = 100 M = 1000 X = fill(rand(), N, M, n) x = mapreduce(r -> sin(r, a, b), +, X, dims = 1) # * What I want * # using ParallelUtilities x = pmapreduce(r -> sin(r, a, b), +, X, dims = 1)