Is there a way to know at any one time in a session how many async threads (if any) are still running and (ideally) be able to recover the corresponding tasks as a variable we can use in a schedule(task, ...)?
I’m aware of current_task() but would ideally like a all_running_tasks() or similar.
Here’s a contrived example:
julia> ping() = while true println("ping"); sleep(5) end
julia> pong() = while true println("pong"); sleep(5) end
julia> @async ping()
julia> @async pong()
# ... ping and pong printed
julia> get_all_running_tasks() # the function I'd like
2-element Array{Task,1}:
Task ...
Task ...
Note: I could write tsk1 = @async ... and tsk2 = @async ... and then inspect those but for the purpose of this question, please assume I can’t do that.
i didn’t find any way ,from what i read in the code, the scheduler is outside julia, so accessing it seems very difficult. an option is to register when a task starts an then finishes the assigned work in a data structure (an array or a dict)
Thanks; I guess I was wondering whether libuv could be coerced to provide this information. E.g. uv_print_active_handles or uv_walk seem promising but I know next to nothing about libuv so would appreciate a more informed opinion + how to call them if appropriate.
i was playing with StackTraces.lookup.(backtrace()) , and it gives a line where the c call to start_task is called, so maybe you can use this to access where and what task was inicialized
I don’t think there’s any official way to do this. There’s the internal Base.Workqueue, but I think this is a temporary holding area for tasks used before they make it into the low level libuv event system.
The internals of this stuff are still changing as the compiler guys work toward better concurrency and there were some PRs merged into 1.2 which change various internals, eg https://github.com/JuliaLang/julia/pull/30838
Thanks Chris, yeah Base.Workqueue is not useful here. I’ve had a look at the PR but I’m not sure it does what I want (but it’s likely I understand it only very partially)
Yes I didn’t mean that PR would help you (sorry). Just pointing out that this stuff is still changing quite a bit internally so the details aren’t settled.
Could I ask about your use case which requires getting a list of tasks? There has been somediscussion of using the emerging ideas of “structured concurrency” (described eg, here) to rework the way that task lifetimes are managed. I wonder if that would help your use case?
Sure, so in HTTP.jl I came accross a sneaky bug whereby calling close(server) (the recommended way to switch off a @async HTTP.listen ... loop) does in fact leave @async tasks running potentially until the Julia session is terminated because they are left unaware of the fact that the server was closed.
I suggested a simple fix for this but it made me realise that this could potentially happen in other places where @async threads may be used and where there may not be an explicit check that they’re properly terminated.
Having some way of verifying that at a given point there are no such threads hanging about, or if there are, being able to interrupt them by throwing or scheduling an error would, I think, be great (or, in some future where that exists, call interrupt(task)).
Thanks, this sounds like exactly the kind of problem that structured concurrency is designed to solve. You might find some of the discussion of robust cancellation interesting over at the Trio forum.
As I understand it, the main idea is to scope the lifetime of child tasks along with the function call stack so that tasks started by a function don’t outlive that function. (More generally, to be explicit about the context in which a task is started so that the task doesn’t outlive that context. Trio calls those contexts “nurseries”.)
,from what i read in the code, the scheduler is outside julia, so accessing it seems very difficult. an option is to register when a task starts an then finishes the assigned work in a data structure (an array or a dict)