[ANN] FoldsThreads.jl: A zoo of pluggable thread-based data-parallel execution mechanisms

Package Announcements
, , ,
5

That’s sound awesome! I’m looking forward to learn the solution you come up with.

I’m thinking to execute code like this

@sync begin
    @spawn f()
    g()
end

as

started = Threads.Atomic{Bool}(false)
function wrapper()
    if !Threads.atomic_xchg!(started, true)
        f()
    end
end
token = maybe_schedule(wrapper)
g()
if Threads.atomic_xchg!(started, true)
    wait(token)
else
    # We just "stole back" the child task f. So executing it here:
    f()
end

maybe_schedule would be something that I’d write by, e.g., wrapping ThreadingUltilies.jl and using it in a (somehow) lock-free manner. Importantly, maybe_schedule(wrapper) is OK to fail since we make sure that f would be executed.

Side note: This is not doable for arbitrary f and g because generic tasks in Julia are allowed to communicate. That is to say,

ch = Channel(0)
f() = put!(ch, 1)
g() = take!(ch)

will deadlock unless I make sure f is scheduled via Julia’s task scheduler. This is what I mean by may-happen parallelism.

Is it possible to check the availability concurrently without it being the bottleneck when processing many tasks?

2 Likes