Best way to parallelize for loop with given thread number

New to Julia
1

Greetings,

what is the most efficient way to parallelize a for loop if I only want to use n of N active global threads?

The way I understand it,

Threads.@threads for it in 1:nthreads

will use all available threads which might not be optimal.
I set export JULIA_NUM_THREADS=10 as an environment variable such that when I run my script in VSCode, the REPL will automatically be launched with 10 threads.
In the script however, I would like to be able to say

threads_to_use = 5
my_func(threads_to_use)

where my_func runs an expensive for-loop that should be parallelized.

The AI overlord informed me of so-called Semaphores, but I have not found much documentation about their usage.

Regards!

2

You can use an outer for loop,

Threads.@threads for it in 1:threads_to_use

and put inside it the divided tasks.

3

Use OhMyThreads.jl

Which allows you to set these dynamically

@set begin
            reducer=+
            ntasks=ntasks
        end
1 Like