Multithreaded mapping of an iterator (not a collection)

Well, it depends. For example, if you don’t care about the ordering, you can “broadcast” the work to multiple tasks using a Channel and Threads.foreach. If you can afford somewhat large buffer and/or each function call has more or less have the same execution time, I think it’s possible to write a generic multi-threaded reduction for purely sequential iterators. If you don’t need the generic reduction and just need map in particular, I think the “large buffering” requirement is not critical (as you are going store everything anyway). I just wanted to suggest halve first since it’s much more composable and generic.

(Edit: I think I’ll implement it in ThreadsX soon-ish unless somebody beats me :slight_smile: Map with purely sequential iterators · Issue #161 · tkf/ThreadsX.jl · GitHub)