I would like to run an optimzation with a costly objective function (about 1s execution time) in parallel. The documentation says:
If the objective function is heavily CPU-bound, it’s possible to utilize multiple processors/threads to speed up computations. To enable multi-threading evaluation of the objective function, set parallelization option to :thread in the Options object.
Is it possible to use multiple processes instead of multiple threads? If yes, how?
I would first recommend using multiprocessing. This approach requires no code changes—simply open multiple Windows terminals or VS Code instances to execute your Julia code. It eliminates the need for code maintenance.
When using multiprocessing, disable parallelism for all libraries. Set linear algebra libraries like BLAS to single-threaded mode. If using HiGHS, configure it to use no more than two threads.
This will not work. An optimizer calls the cost function. I cannot convince the optimizer to open multiple terminals. It should use DistributedNext.jl and the addprocs() function.
For multiprocess, I think we can keep parallelism entirely behind the batched fitness hook (value!).
Evolutionary.jl already has a batched value! hook. We’d “add” a new Options.parallelization mode (e.g. :distributed) and implement its value! method using multiprocess evaluation, while keeping the same in-place contract.
Then you can choose the backend (pmap, a worker pool, etc.) without changing the GA loop.
You can do it with CMAEvolutionStrategy, by setting parallel_evaluation = true and leaving the default value of false for the multi_threading keyword argument. From the docstring for minimize:
If parallel_evaluation = true, the objective function f receives matrices
of n rows (n = length(x0)) and popsize columns and should return a vector of
length popsize.
So your objective function is responsible for distributing the popsize objective vectors to the parallel workers and collecting up the results.