Producer/Consumer Tasks Do Not End Properly

Specific Domains Julia at Scale
1

I want to construct an unbuffered channel and bind a producer task feeding the channel for some certain amount of data and a consumer task consuming the produced data in in infinite loop until a poission pill is taken. The script for this problem is given below.

function producer(ch)
    for i = 1 :  10
        put!(ch, i)
    end
end

function consumer(ch)
    while true
        val = take!(ch)
        val === NaN && break
        println("Took ", val)
    end
end


channel = Channel(0)
producer_task = @task producer(channel)
schedule(producer_task)
yield()
consumer_task = @task consumer(channel)
schedule(consumer_task)
yield()
put!(channel, NaN)  # Poission pill.
println(consumer_task)

The problem is that when I run the whole script at once using ctrl+shift+enter in Juno, I got the following console output,

Took 1
Task (done) @0x00007f4039ad4a90

which implies that the tasks do not work properly. However, when I execute the same script line by line using ctrl+enter in Juno, I get the following console output,

Took 1
Took 2
Took 3
Took 4
Took 5
Took 6
Took 7
Took 8
Took 9
Took 10
Task (done) @0x00007f403a734550

which implies that the tasks work properly. But if I insert a short sleep time before the poissson pill as shown below

# see the full script above
yield()
sleep(eps())
put!(channel, NaN)  # Poission pill.
# see the full script above

the whole script works properly using ctrl+shift+enter.

I can undertand that certain amount of time is required before poission pilling the consumer task but what is the reason of this required time. (I assume that this time is required for the scheduler to do the things right.). And, is there any other possible way to terminate the consumer task harmlessly( i.e. not abruptly closing the channel)?

By the way I realized that minimum sleep time is 1 milisecond. So a sleep of eps() time is not possible.

2

It’s an interesting but slightly confusing case because println causes a yield, so your consumer is yielding partway through execution. A slightly easier-to-understand example might be

function producer(ch)
    for i = 1 :  10
        put!(ch, i)
    end
end

const output = []
const broken = Ref(false)
function consumer(ch)
    while true
        val = take!(ch)
        val === NaN && (broken[] = true; break)
        push!(output, val)
    end
end

in one file (so you can load your function definitions once and don’t have to wonder about compilation time, etc) and

empty!(output)
broken[] = false
channel = Channel(0)
producer_task = @task producer(channel)
schedule(producer_task)
yield()
consumer_task = @task consumer(channel)
schedule(consumer_task)
for i = 1:6
    yield()
end
put!(channel, NaN)  # Poission pill.
println(output)
println(consumer_task)
println(output)
println(broken)

You’ll see that however many loops are taken, that’s the number of elements in output.

3

@tim.holy thank you for your informative reply. That solved the problem.