Yield in iterator

General Usage
1

How can I use a “yield” inside a function like a generator/iterator?

function results()
yield 1
yield 2
yield 3
}

2

Unlike in Python, there is no “yielding iterator” built into Julia. yield in Julia means the currently running task yields to the scheduler, so that other tasks can run.

2 Likes
3

Hi! Do you want to mimic the behavior from Python? E.g.

def results():
    yield 1
    yield 2
    yield 3

Julia doesn’t support the same syntax as far as I know. You could write the same thing as

(i for i in 1:3)

and more complicated things as

(
begin
   ...
end
for i in ...
)

Not sure if there are other short ways of manually creating generator objects, besides the comprehension syntax? It’s a bit less convenient than the Python syntax in my opinion, but one can write a lot of generators like this as well.

The yield function does something else as @Sukera already mentioned.

1 Like
4

There is JuliaDynamics/ResumableFunctions.jl: C# style generators a.k.a. semi-coroutines for Julia. (github.com), but it is not ideal for performance critical tasks.

6 Likes
5

True, but if the comparison point is Python’s yield this likely isn’t a big deal

3 Likes
6

Breaking this down one step at a time. Julia is not Python, terms mean different things, and analogous concepts aren’t one-to-one.

  1. Julia and Python share generator expressions, but there is no generator function concept in base Julia.
  2. Iteration is done differently; Julia’s iteration works on an often unmutated iterable and an iteration-wise state (state is hidden in for-loop), while Python’s iteration works on a stateful iterator created from an often unmutated interable or generator function. Julia does have mutable stateful iterables, and they often would contain the true state while the separate iteration-wise state is reduced to nothing.
  3. Python’s generator functions, coroutines, yielding, etc are a little muddled, and it’s because they are related in a progressive history of asynchronous IO. Julia divides these concepts differently, more starkly. Iteration is done with the iteration interface, asynchronous Tasks are resumable, and data is shared between Tasks via captured Channels queues.
  4. Channel can take a Task that interacts with itself, which is probably the closest in base Julia you can get to the stateful generator function in Python. Note that Python generator functions do not interact with its task scheduler, and the Channel’s Task runs immediately and pauses once it puts something in the channel, so the iteration is offset a bit from the Python example.
julia> function results()
         Channel{Int}(function (channel) # schedules a task
           for i in 1:3
             put!(channel, i); println("round $i")
           end
         end)
       end
results (generic function with 1 method)

julia> r = results()
Channel{Int64}(0) (1 item available)

julia> iterate(r, nothing) # take!(r) is idiomatic
round 1
(1, nothing)

julia> iterate(r, nothing)
round 2
(2, nothing)

julia> iterate(r, nothing)
round 3
(3, nothing)

julia> iterate(r, nothing) # take!(r) errors like Python here
  1. To avoid the task scheduler, you can also mimic Python generated functions with really ugly stateful Julia iterables. The approach below is unadaptable to generator function syntax and requires making several definitions. ResumableFunctions.jl and similar packages have macros that transform function syntax into even more elaborate definitions, but at least you don’t have to write them yourself.
julia> mutable struct Results  state::Int  end

julia> Results() = Results(0)
Results

julia> function Base.iterate(results::Results, state=nothing)
         results.state += 1
         if results.state <= 3
           println("round $(results.state)"); return (results.state, nothing)
         else
           return nothing
         end
       end

julia> r = Results()
Results(0)

julia> iterate(r, nothing)
round 1
(1, nothing)

julia> iterate(r, nothing)
round 2
(2, nothing)

julia> iterate(r, nothing)
round 3
(3, nothing)

julia> iterate(r, nothing)
4 Likes