Happy to announce SurrogateModelOptim.jl a Radial Basis Function based optimization package. It is intended for smooth and expensive functions. Expensive in this case is functions that evaluates in several minutes to hours with a small number of total evaluations.
Out of convenience one can optimize a function by providing the function and search range.
julia> using SurrogateModelOptim
julia> rosenbrock_2D(x) = (1.0 - x[1])^2 + 100.0 * (x[2] - x[1]^2)^2
julia> search_range=[(-5.0,5.0),(-5.0,5.0)]
julia> smoptimize(rosenbrock_2D, search_range)
By default, 20 surrogate models are fit to the same data where the median of these are used as the predicted value. To reduce the cost of creating several surrogates it is recommended to do it in parallel.
The package is based around three exported functions, scaled_LHC_sampling_plan, surrogate_model, model_infill. These let you create a sampling plan, build a surrogate model and then add infill points to your desire. This is probably the most useful functionality as you are free to create constraints, perform multi-objective optimization or otherwise customise the use case. The surrogate model hyperparameters: RBF types, RBF widths, dimension scaling and noise level is by default optimised using BlackBoxOptim.jl so that no prior assumptions about the data need to be made.
It has shown good performance (function value per iteration) against Nelder-Mead from Optim.jl and a Differential Evolution algorithm from BlackBoxOptim.jl for several benchmark cases, both noisy and noise-free functions. It was also briefly benchmarked against BayesOpt.jl with the following settings for both packages:
result = smoptimize(rosenbrock_2D, search_range;
options=SurrogateModelOptim.Options(
iterations=30,
num_start_samples=5,
smooth=false,
infill_funcs=[:median,:std]
));
using BayesOpt
config = ConfigParameters()
config.n_iterations = 30
config.n_init_samples = 5
config.n_iter_relearn = 1
config.noise = 1e-12
optimizer, optimum = bayes_optimization(rosenbrock_2D, first.(search_range), last.(search_range), config)
This was re-run 20 times each. SurrogateModelOptim.jl achieved a mean optimum of 0.01 and BayesOpt.jl achieved a mean optimum of 0.55. Keep in mind that the maximum function value is 90036 so they are both close to the minimum value of 0. This benchmark should be taken with a grain of salt as it was only run with these settings for this one function. In terms of time spent, BayesOpt.jl ran one test in less than a second for this example whereas SurrogateModelOptim.jl took several minutes.
For expensive functions e.g. aerodynamic simulations, the time spent creating the surrogate is small in comparison to the function evaluation. Check the docs for more information. A paper with all the details and benchmark information is in the works.
Special shout-out to @eljungsk for ScatteredInterpolation.jl and the work that was put into the paper and package functionality.
If you find any bugs or have feature requests, please submit an issue and I’ll do my best.