Multi-threaded producer - consumer with threads for leading data

[jling]

Example of Async/Distributed loading into a single shared Channel

I think this is a fairly common senario: You have a collection of files, and each file contains a set of records (e.g. one JSON entry per line). And you which to kinda lazily load up those records, without caring about maintaining order… Possibly you want to do something with the records like filter some of them out or transform them etc. You want to do it lazily because you maybe can’t (or don’t want to) hold all the data in memory at once, or maybe you just want to be able to test your a…