BlackBox Optimization: Multiple Cores (Workers) Rather than Threads

General Usage
, ,
1

The example for blackboxoptim in parallel uses multiple threads (BlackBoxOptim.jl/multithreaded_optimization.jl at master · robertfeldt/BlackBoxOptim.jl · GitHub)

I am interested in using the same approach but on a shared server where I only have one thread but access to many cores (workers).

Is there a way to adjust blackboxoptim to use multiple workers rather than multiple threads?

2

What do you mean when you say you only have access to one thread on your shared server? If you have access to multiple CPU cores then you should be able to use julia’s threaded parallelization.

I think you might be mixing up shared memory parallelization using OS threads, which is what julia’s Base.Threads module does, and CPU hyperthreads.

In parallel programming, threading usually refers to parallelization within a single OS process containing multiple OS threads, whereas distributed memory computing uses multiple processes. How these OS threads or processes map to physical or logical CPU cores is a separate matter. Confusion sometimes arises because modern CPUs often have multiple logical cores, which are sometimes referred to as CPU threads. Here’s a good stack exchange post that gives a brief explanation of the difference between physical and logical cores:

https://unix.stackexchange.com/questions/88283/so-what-are-logical-cpu-cores-as-opposed-to-physical-cpu-cores

2 Likes
3

If you don’t need a specific method of BlackBoxOptim.jl but would be fine with CMA-ES you can use it with option parallel_evaluation. The package should be registered in the next few days.

4

Thanks for the feedback–your hypothesis was correct about getting mixed up between physical vs logical cores.

What I don’t understand, then, is why NThreads=Threads.nthreads() on the shared server, I only see one! It seems like I should see many, as I know the shared server has many cores (and a friend is running parallelized python code on the same server)!

5

By default, julia only runs single threaded. You need to set the number of julia threads you want to run by setting the JULIA_NUM_THREADS environment variable before launching julia. For example, on Linux you can do the following:

$ julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.4.1 (2020-04-14)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> Base.Threads.nthreads()
1

julia> exit()
$ JULIA_NUM_THREADS=4 julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.4.1 (2020-04-14)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> Base.Threads.nthreads()
4

See the multithreading section of the julia manual for more info.

Personally I think it would be nice to be able to change the number of julia threads on the fly, programmatically, but right now that’s not the case.

2 Likes