From what I can tell, CachingPool caches things at the level of the closure, so if you have multiple closures using the same “big” object, it gets sent twice. E.g.
function foo(pool) dat = BigObject(...) pmap(workers()) do i something(dat) end pmap(workers()) do i something_else(dat) end end pool = CachingPool(workers()) foo(pool) # dat will be sent twice
Does anything exist that would cache at the level of the object, so that
dat is only sent once? I’m aware that auto-global-shipping would work, but is there a solution that doesn’t rely on a global state?