# Optimizing noisy objective

**URL:** https://discourse.julialang.org/t/optimizing-noisy-objective/25260
**Category:** Optimization (Mathematical)
**Tags:** question
**Created:** [June 13, 2019, 5:39pm UTC](https://discourse.julialang.org/t/optimizing-noisy-objective/25260 "2019-06-13T17:39:18Z")
**Posts on this page:** 1
**Showing post:** 35

<div class="post-metadata">

### Author: ![asprionj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/asprionj/32/6856_2.png) [@asprionj](https://discourse.julialang.org/u/asprionj)
#### Post date: [June 16, 2019, 8:58pm UTC](https://discourse.julialang.org/t/optimizing-noisy-objective/25260/35 "2019-06-16T20:58:04Z")

</div>

Just some ideas:

- Particle-Swarm Optimisation: Had some good results on various “black-box” problems with this, but probably will require too many iterations. (Also can’t point at any reference implementation right now…)
- DIRECT (e.g. [https://github.com/npinto/direct](https://github.com/npinto/direct))
- Some sort of surrogate modelling ([Wikipedia](https://en.wikipedia.org/wiki/Surrogate_model))?
- Sobol sampling and Gaussian Process Model (GPM) as surrogate: Did this once for a parameter optimisation that used a very expensive simulation (hours per run) as evaluations. We just sampled the parameter space using N Sobol points (we could obtain over night), trained a GPM and figured out if the model accuracy was already stagnating (by cross-validation). Went on with sampling (overnight) until the model accuracy was not improving anymore, then applied a (very expensive) global optimisation algorithm to the GPM which evaluated lightyears faster than the original simulation… don’t know whether this could be successfully applied to such high-dimensional problems though. Maybe sparse GPM’s or a simpler type of surrogate model?
- Polynomial Chaos Expansion (PCE, as surrogate model): according to a colleague of mine, PCE yields reliable (accurate) models with even less samples than e.g. GPM’s. They use it for surrogate modelling of astronomic processes… Just some pointers I found by a quick google:
  - [Polynomial chaos expansion for surrogate modelling: Theory and software](https://www.researchgate.net/publication/327594993_Polynomial_chaos_expansion_for_surrogate_modelling_Theory_and_software)
  - [Data-driven polynomial chaos expansion for machine learning regression](https://www.ethz.ch/content/dam/ethz/special-interest/baug/ibk/risk-safety-and-uncertainty-dam/publications/reports/RSUQ-2018-005.pdf)

---

_[View the full topic](https://discourse.julialang.org/t/optimizing-noisy-objective/25260)._