Call function every N seconds

General Usage
1

In Julia 1.x, is there a way to call a function every N seconds? Some details:

  • reasonable delays in function calls are ok
  • no task cooperation can be assumed, caller is allowed to run entirely within a single task without ever yield-ing
  • if it matters, the function wraps a subroutine in a C library
2

Just a probably very dumb question: why not a dead simple while true loop with a timer?

1 Like
3

Sorry, I wasn’t clear enough - this functions should not stop execution of the main program flow. I work on a library (RDKafka.jl) that let’s you produce messages to a broker, but currently requires regular calls to kafka_poll() function to receive message delivery reports. I want to automate polling in background without stopping the main program flow.

In C or Java I would simply create a thread sleeping 99% of time, but in Julia we only have cooperative multitasking and @threads for ....

4

I think you’re looking for Timer.

3 Likes
5

I believe you are talking about something like this:

t = Timer((t) -> println("running"), 1; interval=1)   # print every 1 second

When running from REPL it indeed works. But REPL let’s scheduler switch between tasks. If we start a long-running not yield-ing task after the timer is created, the scheduler will wait for this task to finish before launching the timer callback:

t = Timer((t) -> print("\nrunning\n"), 1; interval=1)
for i=1:1000
    rand(1000, 1000) * rand(1000, 1000)
    # print(*)
end

On my computer timer starts printing the message only after 35 seconds. Notably, if I uncomment print(*) in the loop, Julia let’s scheduler switch tasks and timer works as expected again.

2 Likes
6

You would definitely need multithreading for this to work, since there’s no other (safe) way to do what you want with the regular libuv cooperative scheduler. The new PARTR threading facilities are probably what you’ll want to use; I won’t have time to put together an MWE today, but do check out the merged PRs with the “multithreading” tag on the julia Github to see some examples of how to use it.

1 Like
7

I was about to write the same message at Julian. There is no safe way to do this without real threads. So if you can live with master (ie, 1.3), that is best.

You could create a thread in C, and do the poll on that thread, but even then doing a callback into Julia with the result will need the main thread to yield.

Regards

Avik

2 Likes
8

Since I’ve been experimenting with related features, I’ll post something which works on yesterday’s nightly (with PARTR):

using Base.Threads
const ctr=Ref(0)
const amidone=Ref(false)
function upd(t)
  ctr[] += 1
end
t = Timer(upd, 1; interval=1)
function periodicjob()
    while isopen(t)
        wait(t)
        c = ctr[]
        println("running $c")
    end
end

function crunch()
    for i=1:1000
          rand(1000, 1000) * rand(1000, 1000)
    end
    println("loop done")
    amidone[] = true
end

if nthreads() == 1
    crunch()
else
    @threads for j=1:2
        id = threadid()
        if id == 1
            tHdl = @task periodicjob()
            schedule(tHdl)
        else
            cHdl = @task crunch()
            schedule(cHdl)
        end
    end
end

# N.B.: controller must stay alive
while !amidone[]
    sleep(0.1)
end
close(t)

IIUC, it is important that the thread 1 code yield (or be trivial). The periodicjob may need to run on thread 1 (or on the thread where its Timer lives?) for consistent timing.

Experiments with more substantial subsidiary tasks suggest that the overhead is modest, but I haven’t thought of a good way to verify that the primary (crunch) task is truly continuous (when BLAS is limited to one thread).

1 Like
9

If your main thread is not doing any I/O, it’s usually not too hard to call yield() every once in a while… it’s mainly problematic if you are spending large blocks of time in external libraries that you can’t modify.

10

Since it’s a library code, I have no control over user’s code is doing. Although I believe in practice it won’t be a problem in most cases.

11

Thanks everyone, I’m excited to see new threading functionality in Julia 1.3. However, to maximize adoption of the library I think I will stick to @tkoolen solution based on timers and add caution to the README. Later we will be able to use VERSION to dispatch between the current hack and appropriate thread-based implementation.