Here is a quote from the README:
Transducers.jl provides composable algorithms on sequence of inputs.
They are called transducers, first introduced in Clojure language
by Rich Hickey.Using transducers is quite straightforward, especially if you already
know similar concepts in iterator libraries:using Transducers xf = Partition(7) |> Filter(x -> prod(x) % 11 == 0) |> Cat() |> Scan(+) mapfoldl(xf, +, 1:40)
However, the formalization of the transducers is quite different from
iterators and resulting in a better performance for complex
compositions.See more in the documentation.
Installation
]add https://github.com/tkf/Transducers.jl
There are more examples in the documentation (see also reference manual). I also discussed difference to iterators in the documentation.
Some (technical) highlights:
- It turned out the composed/lowered code is quite compiler-friendly. For example,
julia
generates SIMD instructions forFilter(...) -> Map(...)
without me explicitly coding for SIMD. - āNested loopā construct like
Cat
(concatenation) is quite easy to write (and efficient). No state juggling as in iterators. Itās a good fit for the split-apply-combine pattern. - This also applies to the container types. If you have a container type that needs nested for loops, it becomes non-trivial to write
Base.iterate
while supporting transducers is quite straightforward. See also: How to make your data type reducible. - A subset of transducers support parallelism (based on
Base.Threads
). Here is an example of splitting a string into words and counting them in parallel, partially based on Guy Steeleās 2009 ICFP talk (which is a good talk to watch, by the way).
Implementing transducers in a Julia-friendly way was an interesting holiday project. It even makes me wonder why itās not as main stream as iterators (outside Clojure world?). I know there is a C++ library which is actually where I get the idea of type-stability consideration. Anyway, if you have any thoughts on this, Iād like to hear.
Wish you a happy new year!