I am addicted to those yield statements in python. If you’re in the same boat checkout PyGen. I know you can achieve the same thing with Channels but its always felt a little clunky to me. Anyways, here is how it works:
julia> using PyGen
julia> @pygen """
function fibonacci()
n, m = 0, 1
while true
yield m
n, m = m, n + m
end
end
"""
fibonacci (generic function with 1 method)
julia> for i in fibonacci()
println(i)
sleep(1)
end
1
1
3
5
8
.
.
.
Enjoy 
1 Like
Cool! Have you considered writing a regular macro, instead of one operating on strings?
@pygen function fibonacci()
n, m = 0, 1
while true
yield(m)
n, m = m, n + m
end
end
will be more robust, and won’t kill auto-indent/syntax highlighting. You might find MacroTools’ expression walking functions useful.
7 Likes
Oh good call, I did it this way because the REPL would barf as soon as I wrote any yield x statements, but the yield(x) fixes that. MacroTools looks heavenly, thanks for the advice
If you don’t want to use yield(m) but have more of a statement feel, you can use @yield m. You don’t ever have to define a @yield macro if you make sure to handle and eliminate those @yield Exprs in the pygen macro.
6 Likes
Thanks for the tips everyone! @pygen is even better than before :).
julia> using PyGen
julia> @pygen function pascal(n)
i = 0
while i <= n
yield(i)
i += 1
end
end
pascal (generic function with 1 method)
julia> n = 20;
julia> sum(pascal(n)) == n * (n + 1) / 2 # Booyah computers work
true
I think I’m beginning to see the light on the whole macro business… Feels good to just add the features you want to a language.
6 Likes
The name “PyGen” suggests that it is something to do with Python, whereas it isn’t really. Maybe something like “yieldgen”?
4 Likes
I suppose as a package name it sort of clashes with the current trend of being a wrapper of a Python library or Python itself, but I don’t see why implementing a piece of syntax from Python is very different. Unfortunately, the vocabulary on the Julia side makes hard to talk about it without reference to Python.
@yield, @generator etc already have specific meaning in Julia… perhaps @pyyield because it supplies the Python type yield statement?
I do agree it has a troubled sort of name (both the macro and the package).
(ps: 0.5 is supported now, sorry about that!)
It is also worth pointing out that this syntax has been around in C# for a very long time. Not clear to me whether Python or C# introduced it, but in any case, that would also support a name that doesn’t have “Python” in it.
I actually really wish julia had this build in, like Python and C#…
2 Likes
Oh, fair enough! Yeah, seems especially narrow to call it Py-anything then doesn’t it.
I have to agree about wishing this idiom was in julia natively.
function f(x)
...
yield y
...
end
is much more clean than the alternative:
function f(x)
function temp(c)
...
push!(c, y)
...
end
return Channel(c -> temp(c))
end
Though, that might just be my lack of creativity…
3 Likes
FYI, Channel(c -> temp(c)) can be simplified to just Channel(temp). Even better, with do-syntax:
function f(x)
return Channel() do c
...
push!(c, y)
...
end
end
6 Likes
And even better (in my opinion), combining it with short-form function syntax,
f(x) = Channel() do c
...
push!(c, y)
...
end
Effectively a macro-free way of defining a python-like generator.
11 Likes
Yeah, I think I have to agree. I’ve always had reservation about the do-syntax because I find it reads a little weird, but this statement as really great. Its very clear that you’re filling up a channel with values.
1 Like
Ah, I might have misunderstood the Python use of yield. I would like the C# version of yield, which simply is a shortcut to implementing an iterator, without the overhead of Channel etc. That is what I would really like to see in julia.
2 Likes
You’re partially correct about the python yield statement from what I understand. They construct a generator which is a type of iterator, but I think that iterator is implemented with a type of coroutine, similar to the julia Task. The @pygen macro implements the closest julia equivalent I can find: a Task that communicates over a Channel in v0.6, or previous to v0.6 a Task that communicates via produce() and consume().
Now, if there was a way to compile a function with yield statements into a nice high performance iterator (sounds like c# does this?) that would be amazing… A brief play with these Channels shows they’re no competition for an iterator when it comes to performance.
No I don’t think it’s similar to Task.
You can see what they actually are here under the section “How Python Generators Work”. I couldn’t find much detail on the Julia’s Task implementation but looking at the source is smells pretty similar (a stack frame + pointer to the instruction we’re at currently etc).
I am kind of curious about what C# is doing though, just because some calls itself an “iterator” doesn’t preclude the same co-routine stuff to be happening in the background I suppose.
It seems that C# got away without a stack. They just expand the whole thing to in the callers stack as an object carrying the state and a bunch of goto statements to hop from where the iterator consumed and back again. Article here.