Threads.@threads to return results

Internals & Design
1

This is a request for “enhancement” I guess. It would be nice if
`Threads.@threads returned the results. Something like:

v1, v2, v3 = Threads.@threads for w in [f1, f2, f3]
  w()
end
2

For loops don’t return a value…

2 Likes
3

if you are asking if that’s possible, it’s not.

It think something like this would work.

v = Vector{SomeType}(undef, 3)
fs = [f1, f2, f3]
Threads.@threads for i in 1:length(fs)
  v[i] = w(fs[i])
end
4

This should not be added to Threads.@threads since the macro has no way of knowing if there’s a return value. It’s obviously also inconsistent with normal for loop.

What you are looking for is a multithreaded map. I don’t think Base/Stdlib has it yet (though I didn’t check lately) but I’m pretty sure there are many packages that provide a similar API.

5

I think this should do what you want:

v1, v2, v3 = fetch.(Threads.@spawn w() for w in [f1, f2, f3])

In many use cases, the spawn syntax is superior to @threads because it uses dynamic scheduling, e.g. is efficient also when the function run times are vastly different.

1 Like
6

You’d need the parentheses around Threads.@spawn w() as in ((Threads.@spawn w()) for w in [f1, f2, f3]).

FYI, @threads uses the mechanism same as @spawn since Julia 1.5. Though of course it’s true that @spawn is more flexible and @threads is not as dynamic as it could be.

For a more flexible @threads-like for loop syntax, see GitHub - JuliaFolds/FLoops.jl: Fast sequential, threaded, and distributed for-loops for Julia—fold for humans™

For parallel map and other Base functions, see GitHub - tkf/ThreadsX.jl: Parallelized Base functions

For other threaded map implementations, see:

5 Likes
7

For me, the code works also without the parenthesis (tested in Pluto with Julia 1.5.1 before).

8

Yes, it works, but it will only spawn one task, not three, because without the parentheses it’s equivalent to fetch.(Threads.@spawn(w() for w in [f1, f2, f3])).

9

I put sleep(2) in all 3 functions, and the total runtime of the command without the inner parenthesis was only about 2s, the same as if I would add the parenthesis as suggested by @tkf.

10

That’s because the result is lazy, so the functions will only run once you actually access the items in the generator. I am wondering why it still takes 2s for you to run this, perhaps it’s just compilation time, or you are actually accessing the first element for some reason.

11

OK cool! Just the thing.

12

Thanks for references!

13

Ah, sorry, I only checked

julia> (Threads.@spawn identity(w) for w in [1, 2, 3])
ERROR: syntax: unexpected ")"
Stacktrace:
 [1] top-level scope
   @ none:1

and

julia> ((Threads.@spawn identity(w)) for w in [1, 2, 3])
Base.Generator{Vector{Int64}, var"#41#43"}(var"#41#43"(), [1, 2, 3])

But you are right that

julia> identity.(Threads.@spawn identity(w) for w in [1, 2, 3])
3-element Vector{Task}:
 Task (done) @0x00007efe17835b60
 Task (done) @0x00007efe17835cd0
 Task (done) @0x00007efe17835e40

is equivalent to identity.((Threads.@spawn identity(w)) for w in [1, 2, 3]).

2 Likes