Looking for advice on threading

[Jakub_Wronowski]

Search for “map reduce” design pattern.

Parallel reductions

“Parallel mapreduce” is a really common need, and it’s surprisingly hard to find a Julia implementation. Say we have julia> x = randn(10000000); julia> f = log∘sqrt∘abs #56 (generic function with 1 method) The goal is to build a fast, parallel work-alike to julia> @btime sum($f.($x)) 150.037 ms (2 allocations: 76.29 MiB) -3.1741049251367934e6 julia> @btime mapreduce($f,$+,$x) 135.280 ms (0 allocations: 0 bytes) -3.1741049251367934e6 When I asked this on Julia Slack #multithreading, @a…

Map and mapreduce with Threads

I would like to experiment with parallelizing an embarrassingly parallel computation on an iterable using the new multithreading in 1.3. I need map and mapreduce. I would prefer to leave return type determination to Julia, so I would prefer not to preallocate. Do I just need to @spawn and then fetch, as in using Base.Threads: @spawn, fetch, threadid, nthreads @show nthreads() function ploop(f, itr) map(fetch, map(i -> @spawn(f(i)), itr)) end function pmapreduce(f, op, itr) mapreduc…

Anyway 6x on desktop seems to be very good for me.