Task/thread-local caches/buffers

[jling]

How to correctly design multi-thread cache that's transparent to user iteration?

Runnable code and benchmark at this gist. This is a widespread pattern for columnar-storage access, usually each column is too large to fit in ram, so it’s chunked in to clusters, and each cluster is individually compressed on disk. When user getindex(col, idx), you first need to find the cluster, decompress it, and compute the localidx and return user the data. But user is very likely to also immediately getindex(col, idx+1), so you want to cache the result of decompressed cluster. And the …

maybe can provide some inspiration