In Julia 1.x, is there a way to call a function every N seconds? Some details:
- reasonable delays in function calls are ok
- no task cooperation can be assumed, caller is allowed to run entirely within a single task without ever
- if it matters, the function wraps a subroutine in a C library
Just a probably very dumb question: why not a dead simple
while true loop with a timer?
Sorry, I wasn’t clear enough - this functions should not stop execution of the main program flow. I work on a library (RDKafka.jl) that let’s you produce messages to a broker, but currently requires regular calls to
kafka_poll() function to receive message delivery reports. I want to automate polling in background without stopping the main program flow.
In C or Java I would simply create a thread sleeping 99% of time, but in Julia we only have cooperative multitasking and
@threads for ....
I think you’re looking for
I believe you are talking about something like this:
t = Timer((t) -> println("running"), 1; interval=1) # print every 1 second
When running from REPL it indeed works. But REPL let’s scheduler switch between tasks. If we start a long-running not
yield-ing task after the timer is created, the scheduler will wait for this task to finish before launching the timer callback:
t = Timer((t) -> print("\nrunning\n"), 1; interval=1)
rand(1000, 1000) * rand(1000, 1000)
On my computer timer starts printing the message only after 35 seconds. Notably, if I uncomment
print(*) in the loop, Julia let’s scheduler switch tasks and timer works as expected again.
You would definitely need multithreading for this to work, since there’s no other (safe) way to do what you want with the regular libuv cooperative scheduler. The new PARTR threading facilities are probably what you’ll want to use; I won’t have time to put together an MWE today, but do check out the merged PRs with the “multithreading” tag on the julia Github to see some examples of how to use it.
I was about to write the same message at Julian. There is no safe way to do this without real threads. So if you can live with master (ie, 1.3), that is best.
You could create a thread in C, and do the poll on that thread, but even then doing a callback into Julia with the result will need the main thread to yield.
Since I’ve been experimenting with related features, I’ll post something which works on yesterday’s nightly (with PARTR):
ctr += 1
t = Timer(upd, 1; interval=1)
c = ctr
rand(1000, 1000) * rand(1000, 1000)
amidone = true
if nthreads() == 1
@threads for j=1:2
id = threadid()
if id == 1
tHdl = @task periodicjob()
cHdl = @task crunch()
# N.B.: controller must stay alive
IIUC, it is important that the thread 1 code yield (or be trivial). The
periodicjob may need to run on thread 1 (or on the thread where its Timer lives?) for consistent timing.
Experiments with more substantial subsidiary tasks suggest that the overhead is modest, but I haven’t thought of a good way to verify that the primary (
crunch) task is truly continuous (when BLAS is limited to one thread).
If your main thread is not doing any I/O, it’s usually not too hard to call
yield() every once in a while… it’s mainly problematic if you are spending large blocks of time in external libraries that you can’t modify.
Since it’s a library code, I have no control over user’s code is doing. Although I believe in practice it won’t be a problem in most cases.
Thanks everyone, I’m excited to see new threading functionality in Julia 1.3. However, to maximize adoption of the library I think I will stick to @tkoolen solution based on timers and add caution to the README. Later we will be able to use
VERSION to dispatch between the current hack and appropriate thread-based implementation.