Hello Julia Community,
I like to translate a chunked file loader that uses generators to Julia. The idea is to implement it in a most lazy way that triggers the execution and continuation of the costly file-loading only when requested (i.e. on
take!). I have assembled an example (without the file loading) to describe the issue:
The Julia translation of the application looks like this:
println("A ::use it"); # if(somecondition) for x in ch_unbuf println("A ::got ",x); # if(someothercondition); break; end end # end
for a simple “generator”
N = 0; g = function (c::Channel) println(" G::init!"); put!(c,N+=1); println(" G::calc!"); put!(c,N+=1); println(" G::calc!"); put!(c,N+=1); println(" G::end!"); # close(c); # don't need this when using `bind(ch_unbuf,task)` end
I like to achieve the following program flow:
A ::instanciate A ::use it G::init! A ::got 6 G::calc! A ::got 7 G::calc! A ::got 8 G::end!
g task should only run/continue when requested by the first “user” code-snippet.
Observe, how the “generator” acts more like a source that might not run at all depending on
somecondition or might be interrupted by
I got this running with the following setup after a little fiddling around with it. Again: I am interested in the case where a
take! on the channel should block the taking task and schedule the putting task “on the other side”.
So here is what I already got to make the above program flow happen (including the commented-out fiddle):
println("A ::instanciate"); N = 5; # ch_unbuf = Channel(g); # ↯ this does instantly run g one time ch_unbuf = Channel(0); # use an unbuffered `0` channel to trigger execution only when needed # task = @task g(ch_unbuf); # ↯ creates a "runnable" task that won't run instantly, but does not run at all since the task is not scheduled and somehow the for-in loop does not dare to do so either # task = schedule(@task g(ch_unbuf)); # creates a "queued" task that won't run instantly, but will (since its scheduled) run at least once, even if the for-in does not iterate at all task = @async g(ch_unbuf); # this is somewhat the same as the previous line bind(ch_unbuf,task); # this automatically closes the channel when the tasks finishes execution (and therefore terminates the iteration of for … in …)
There are some Issues with the current approach:
It seems that creating the task as a “runnable” one, without scheduling it, and then iterating over with
for … in … is the most elegant way. Unfortunately,
for … in …, when waiting on the channel of that “runnable” task, won’t schedule it (and therefore nothing happens at all). Scheduling the task in the instanciate phase results in executing it at least once (at some time lateron), even if the
for … in … loop does not execute at all (because it might get skipped by
So the only solution I could come up with, is scheduling the
ch_unbuf channels task exactly at the point when it is first used. This is easy to determine for this example, but in a more complex decision tree it becomes more elaborate.
• Is there a better approach to this?
• Shouldn’t a
for … in … loop be able to schedule the task (I assume that it is possible to find out the channels one task to which lifetime the channel is bound and shedule it if its just “runnable” and not “queued”)?
• Or what am I missing here conceptually?