From what I can tell, CachingPool caches things at the level of the closure, so if you have multiple closures using the same “big” object, it gets sent twice. E.g.
function foo(pool)
dat = BigObject(...)
pmap(workers()) do i
something(dat)
end
pmap(workers()) do i
something_else(dat)
end
end
pool = CachingPool(workers())
foo(pool) # dat will be sent twice
Does anything exist that would cache at the level of the object, so that dat
is only sent once? I’m aware that auto-global-shipping would work, but is there a solution that doesn’t rely on a global state?