"Functional State" Iterators & Why isn't this loop as fast as entitlement?

[c42f]

I didn’t read all the code here, but the general issue of how to express efficient iteration kernels in an iterator interface brings to mind the transducer approach as a promising alternative. In particular I thought the following comment about foldl implementations was particularly interesting:

ANN: Transducers.jl, efficient and composable algorithms for map- and reduce-like operations

Another thought: I think the best “feature” of transducers for performance-oriented language like Julia is actually not transducers themselves but rather the foldl implementations specialized for container types. I guess that’s not news since Julia Base has very efficient foldl/mapfoldl. However, foldl (or mapfoldl) itself is not really composable — that’s where transducers come in. Transducers are composable “pre-processors” of the “reducing function” op you passed to foldl. I think one of…

See also the documentation section in Transducers.jl regarding the structure of the lowered loops:
https://tkf.github.io/Transducers.jl/dev/#Difference-to-iterators-1