How to avoid recompiling when using job array on a cluster

I usually use Iterators.product with pmap:

repeats=3
n=10
seeds=rand(n)

wrapped_fn(vals)=run_chain(vals[1])
chains=pmap(wrapped_fn, Iterators.product(seeds,1:repeats))

# chains is n by 3
# can check if the 3 chains in each row are the same in parallel too, up to you

This really helps for doing a grid search of parameter, just like you would with a job array. Hopefully, this is helpful.

2 Likes