Pmap slow compared to map

Hello,

Your issue is similar to mine from last month; basically, closures in pmap are slow when passing large objects (like your 100x100 im1 and im2 arrays). There is a github issue with a bit more detail.

The simplest way to fix the issue seems be using the cache pool:

pool = CachingPool(workers())
pmap(pool, x -> crossCorr_par(im1_shared, im2_shared, x), iterations)

It looks like the CachingPool approach was intended to become default in pmap, based on this pull request from last year, but the proposed change remains under review after a year.

3 Likes