Here is a quote from the README:
Transducers.jl provides composable algorithms on sequence of inputs.
They are called transducers, first introduced in Clojure language
by Rich Hickey.
Using transducers is quite straightforward, especially if you already
know similar concepts in iterator libraries:
using Transducers xf = Partition(7) |> Filter(x -> prod(x) % 11 == 0) |> Cat() |> Scan(+) mapfoldl(xf, +, 1:40)
However, the formalization of the transducers is quite different from
iterators and resulting in a better performance for complex
See more in the documentation.
Some (technical) highlights:
- It turned out the composed/lowered code is quite compiler-friendly. For example,
juliagenerates SIMD instructions for
Filter(...) -> Map(...)without me explicitly coding for SIMD.
- “Nested loop” construct like
Cat(concatenation) is quite easy to write (and efficient). No state juggling as in iterators. It’s a good fit for the split-apply-combine pattern.
- This also applies to the container types. If you have a container type that needs nested for loops, it becomes non-trivial to write
Base.iteratewhile supporting transducers is quite straightforward. See also: How to make your data type reducible.
- A subset of transducers support parallelism (based on
Base.Threads). Here is an example of splitting a string into words and counting them in parallel, partially based on Guy Steele’s 2009 ICFP talk (which is a good talk to watch, by the way).
Implementing transducers in a Julia-friendly way was an interesting holiday project. It even makes me wonder why it’s not as main stream as iterators (outside Clojure world?). I know there is a C++ library which is actually where I get the idea of type-stability consideration. Anyway, if you have any thoughts on this, I’d like to hear.
Wish you a happy new year!