An experiment for Python Style (but Unidirectional) Generators for Julia

Internals & Design
2

I think the style developed in FGenerators.jl combined with FLoops.jl is a more performant and well founded way to go about this, rather than spawning tasks and trying to properly manage the yielding. But it relies heavily on transducers, so that is a lot of infrastructure to digest and learn.

Here’s an example where I believe your use of yieldto fails:

julia> using .PyStyleButUnidirGenerators

julia> @generator function organpipe(n::Integer)
           i = 0
           while i != n
               i += 1
               @yield i
           end
           while true
               i -= 1
               i == 0 && return
               @yield i
           end
       end;

julia> collect(organpipe(2))

I left this on my computer for about 5 minutes and it never completed. I assumer there is some sort of task deadlock. I think if you want to do this with tasks, you’re better off making a channel, and then put!ing data into that channel at each yield, rather than trying to manage the task switching yourself.

Here’s how FGenerators.jl performs with that organpipe:

julia> using FGenerators

julia> @fgenerator function organpipe(n::Integer)
           i = 0
           while i != n
               i += 1
               @yield i
           end
           while true
               i -= 1
               i == 0 && return
               @yield i
           end
       end;

julia> let n = Ref(100)
           @btime sum(organpipe($n[]))
           @btime collect(organpipe($n[]))
       end;
  13.867 ns (0 allocations: 0 bytes)
  1.456 μs (204 allocations: 13.80 KiB)
3 Likes