Nesting `Threads.@thread` and `Polyester.@batch` (or context manager to limit Polyester threads)

How dangerous is it to run Polyester.@batch inside of a Threads.@thread?

An example where I might need this: a library implements a low level function that uses Polyester for multithreading, and I want to run multiple such functions in parallel.

Furthermore, to make this actually useful (i.e., not letting a single Polyester.@batch hogging all the threads), is it possible to control the number of Polyester threads with a context manager of some kind. E.g., how do I implement this pseudocode:

function f(arg)
    println(arg)
    Polyester.@batch for i in 1:10000
        do_something()
    end
end

@thread for j in 1:2
    set_max_polyester_threads(2)
    f(j)
end

I want this code to cause 4 threads to be running in parallel. Two threads dedicated to f(1) and two threads dedicated to f(2). I want this to work in a situation in which I can not modify f itself. Is this possible?

Background: I am using Polyester because benchmarking has shown that cheap threads for the “inner” problem do provide a big performance gain. On the other hand, the “inner” problem can not effective use more than a handful of threads. The outer problem (running the inner problem multiple times) is embarrassingly parallel.

I had some functionality to support something like that in Polyester before, but ripped it out as it wasn’t documented and I didn’t think anyone was using it, and I didn’t feel like maintaining it at the time.

Something like that could be added again, but wouldn’t it be better to just disable inner threading entirely, focusing on outer threading of your embarrassingly parallel program only?

That’d be easier to do without modifying f, and in fact is something you could already do without modifying the library.

t, r = PolyesterWeave.request_threads(PolyesterWeave.num_threads())
# Polyester, LoopVectorization's threading, and Octavian's threading are now disabled
foreach(PolyesterWeave.free_threads!, r)
# they're now reenabled
1 Like

Yes, this perfectly addresses my use case. And you are right about just disabling the inner threads. I guess the memory pressure might be a bit worse if I disable the inner threads, but it is not worthwhile for me to profile that yet.

Also, because this is definitely relying on internals, we could expose something with some guaranteed stability.
Perhaps a function, disable_polyester_threads(f::F).

1 Like

I will make a pull request to Polyester with this.

Here is a draft disable_polyester_threads by Krastanov · Pull Request #86 · JuliaSIMD/Polyester.jl · GitHub