Multithreading and "silent errors"

New to Julia
,
1

Consider the following MWE:

using Base.Threads: @spawn

function f()
    error("Some error")
end

function g()
    while true
        sleep(1)
    end
end

@info "Initialize multi-threaded processing ($(Threads.nthreads()) threads available)."
@sync begin
    @spawn f()
    @spawn g()
end

If I run the above code, the error will occur but remain silent. This is inconvenient for debugging.

In this thread, it has been suggested to work with @macroexpand. Interestingly, if I put @macroexpand in front of the faulty function f(), the error remains silent; if however I put it in front of g(), the error is shown on the CLI.

This leads me to two questions:

  • If I need to debug such a multithreaded code, what is the best way to make errors occurring within tasks visible in the CLI? Indeed using @macroexpand, or are there cleaner ways?
  • If indeed the best way is to use @macroexpand, where exactly should it be placed?
2

@macroexpand only shows you what an expanded macro (which is an AST transform, it replaces syntax with other, custom syntax) looks like. Placing it in front of @spawn only returns what that expanded form looks like, the code isn’t run anymore.

First, @sync and Threads.@spawn are only related insofar they’re both managing Tasks. The former is about tasks in general (though originally intended for single-threaded, asynchronous code using @async, it also works with tasks spawned by @spawn), the latter is for spawning a Task on different threads other than the original one.

Second, @sync captures all exceptions that occur in spawned tasks inside of itself, as documented by its docstring:

help?> @sync
  @sync

  Wait until all lexically-enclosed uses of @async, @spawn, @spawnat and
  @distributed are complete. All exceptions thrown by enclosed async
  operations are collected and thrown as a CompositeException.

So you’d only get the exception once @sync determines that all enclosed tasks have finished. In your example, that wouldn’t ever be the case due to the infinite loop. If you want to get those exceptions earlier, you’ll have to try/catch or wait(task) and handle them appropriately. This may (depending on your usecase) require additional communication, locking and/or synchronization with your long running task, so I’d recommend structuring your long running task with periodic checks for messages in e.g. a Channel, to communicate that it should e.g. terminate, change behavior…

In general though, I’d recommend checking if you can avoid only testing your program in the presence of multithreading. If possible, try to test & find edge cases on a single thread, and only use multithreading when you’re reasonably certain that you’ve handled what can go wrong.

3 Likes