# Parallel Optimization with Surrogates.jl

**URL:** <https://discourse.julialang.org/t/parallel-optimization-with-surrogates-jl/93586>\
**Category:** Optimization (Mathematical)\
**Tags:** question\
**Created:** [January 26, 2023, 4:27pm UTC](https://discourse.julialang.org/t/parallel-optimization-with-surrogates-jl/93586 "2023-01-26T16:27:22Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Dreycen\_Foiles](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dreycen_foiles/32/44744_2.png) [@Dreycen\_Foiles](https://discourse.julialang.org/u/Dreycen_Foiles)\
**Post date:** [January 26, 2023, 4:27pm UTC](https://discourse.julialang.org/t/parallel-optimization-with-surrogates-jl/93586/1 "2023-01-26T16:27:22Z")

</div>

I’m thinking about using Surrogates.jl to optimize a process in my lab using the Gaussian processes for Bayesian optimization. It’s easier for me to run multiple experiments at once, so I was wondering if it was possible to have Surrogates.jl ask for multiple points at the same time? I’m not seeing any obvious settings in the documentation.

If it’s not possible to do with Surrogates.jl, are there any other packages that would have this functionality?

Thank you for your time

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [January 26, 2023, 7:19pm UTC](https://discourse.julialang.org/t/parallel-optimization-with-surrogates-jl/93586/2 "2023-01-26T19:19:04Z")

</div>

> [@Dreycen\_Foiles](#):
>
> I’m thinking about using Surrogates.jl to optimize a process in my lab using the Gaussian processes for Bayesian optimization. It’s easier for me to run multiple experiments at once, so I was wondering if it was possible to have Surrogates.jl ask for multiple points at the same time? I’m not seeing any obvious settings in the documentation.

Just give the constructor an array. Most of the tutorials show this for each specific surrogate type.

---

<div class="post-metadata">

**Author:** ![platawiec](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/platawiec/32/31914_2.png) [@platawiec](https://discourse.julialang.org/u/platawiec)\
**Post date:** [January 26, 2023, 9:04pm UTC](https://discourse.julialang.org/t/parallel-optimization-with-surrogates-jl/93586/3 "2023-01-26T21:04:11Z")

</div>

With respect to Surrogates.jl, there should be no problem getting multiple points at once from a Gaussian Process surrogate, as Chris suggests.

However, serial Bayesian optimization won’t be able to give you which points to query, as you need to optimize over the joint distribution of multiple queries. For that, you will need something like [Botorch](https://botorch.org/), specifically see [Batching · BoTorch](https://botorch.org/docs/batching) and [Optimization · BoTorch](https://botorch.org/docs/optimization#joint-vs-sequential-candidate-generation-for-batch-acquisition-functions)

Unfortunately I’m not aware of anyone porting the algorithms to Julia.

---

<div class="post-metadata">

**Author:** ![Dreycen\_Foiles](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dreycen_foiles/32/44744_2.png) [@Dreycen\_Foiles](https://discourse.julialang.org/u/Dreycen_Foiles)\
**Post date:** [January 26, 2023, 10:35pm UTC](https://discourse.julialang.org/t/parallel-optimization-with-surrogates-jl/93586/4 "2023-01-26T22:35:46Z")

</div>

Thank you, for your help! I apologize because I think I was unclear with my question. I see now that it’s pretty simple to get multiple points out of an already constructed surrogate model. However, I don’t see how it’s possible construct a surrogate model in parallel. For example, here is a Kriging example from the documentation.

```julia
using Surrogates
f = x -> exp(x)*x^2+x^3
lb = 0.0
ub = 10.0
x = sample(50,lb,ub,UniformSample())
y = f.(x)
p = 1.9
my_krig = Kriging(x,y,lb,ub,p=p)

#I want an approximation at 5.4
approx = my_krig(5.4)

surrogate_optimize(f,LCBS(),lb,ub,my_krig,UniformSample(); maxiters = 10, num_new_samples = 10)

```

I realize that if I wanted to query my pre-existing model, I could just define an array `arr=[1,2,3]` and get the value of the surrogate at those points with `my_krig.(arr)`. However, I’m interested in querying multiple points during the optimization process itself. How could I get `surrogate_optimize` to query `f` multiple times before refitting the surrogate model and asking asking for the next batch of points?

It seems like @platawiec is saying this is currently not possible. Looking more into it, I saw this issue on the Surrogates.jl repo: [Plans for parallel optimization · Issue #278 · SciML/Surrogates.jl (github.com)](https://github.com/SciML/Surrogates.jl/issues/278). It doesn’t seem too terribly complicated but it was never resolved, so did it end up being harder than originally thought?

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [January 27, 2023, 10:30am UTC](https://discourse.julialang.org/t/parallel-optimization-with-surrogates-jl/93586/5 "2023-01-27T10:30:23Z")

</div>

> [@Dreycen\_Foiles](#):
>
> It seems like @platawiec is saying this is currently not possible. Looking more into it, I saw this issue on the Surrogates.jl repo: [Plans for parallel optimization · Issue #278 · SciML/Surrogates.jl (github.com)](https://github.com/SciML/Surrogates.jl/issues/278). It doesn’t seem too terribly complicated but it was never resolved, so did it end up being harder than originally thought?

It’s not hard, someone just has to do it.

---

<div class="post-metadata">

**Author:** ![Dreycen\_Foiles](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dreycen_foiles/32/44744_2.png) [@Dreycen\_Foiles](https://discourse.julialang.org/u/Dreycen_Foiles)\
**Post date:** [January 27, 2023, 5:31pm UTC](https://discourse.julialang.org/t/parallel-optimization-with-surrogates-jl/93586/6 "2023-01-27T17:31:40Z")

</div>

Okay, then I’ll give it a try, myself.

---

<div class="post-metadata">

**Author:** ![Dreycen\_Foiles](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dreycen_foiles/32/44744_2.png) [@Dreycen\_Foiles](https://discourse.julialang.org/u/Dreycen_Foiles)\
**Post date:** [June 14, 2023, 8:08pm UTC](https://discourse.julialang.org/t/parallel-optimization-with-surrogates-jl/93586/7 "2023-06-14T20:08:36Z")

</div>

I finally got around to working on this. I’ve opened up a [draft PR](https://github.com/SciML/Surrogates.jl/pull/435) that introduces the beginnings of this functionality. It’s definitely not finished yet, but I’ll keep working on it once I get the go ahead that my current approach is promising.