I have a f: \mathbb{R}^n \to \mathbb{R}^m function, m > n, and I am trying to minimize some norm of f(x). The norm itself is flexible, I would like to get *something* working.

The problem is that f is stochastic as it comes from a simulation. I have tried to reduce variance in the obvious ways (common random variates) where I could, but some remains. The original problem is an indirect inference exercise (in economics).

Any advice on methods and their tuning would be appreciated.

I boiled it down to the following MWE:

```
const Ļ = [0.000262; 0.000316; 0.000638; 0.00119; 0.00118;
0.00126; 0.0014; 0.000233; 0.000243; 0.000291]
const J = [0.23 -0.194 0.672 1.12 -0.0157 -0.0287;
0.225 -0.187 0.655 1.15 -0.00563 -0.0471;
0.193 -0.107 0.615 1.24 0.0423 -0.0476;
-0.0154 0.216 0.141 1.52 0.0216 -0.0649;
-0.0195 0.186 0.0519 1.76 0.261 -0.0149;
-0.011 0.184 0.0554 1.96 0.483 0.102;
0.0042 0.174 0.0502 2.07 0.614 0.132;
0.0223 0.000494 -0.0088 0.000412 -0.003 0.0623;
-0.00222 0.0721 -0.0233 -0.00147 -0.00651 0.0187;
0.0087 0.00169 -0.00492 0.000407 -0.00184 -0.045]
f(x) = J * x .+ (randn(10) .* Ļ)
g1(y) = sum(abs, y) # norm
g2(y) = sum(abs2, y) # other norm?
lo, hi = fill(-5.0, 6), fill(5.0, 6) # bounds if needed
```

This is an approximate problem derived from the real one, for the MWE. For the actual `f`

, evaluation takes about 2s.

Here is what I tried so far:

# BayesOpt

@jbreaās excellent BayesOpt gives the best solution:

```
using BayesOpt
config = ConfigParameters() # calls initialize_parameters_to_default of the C API
set_kernel!(config, "kMaternARD5") # calls set_kernel of the C API
config.sc_type = SC_MAP
config.n_iterations = 500
config.init_method = 3
config.n_init_samples = 100
config.n_iter_relearn = 1
optimizer, optimum = bayes_optimization(g1 ā f, lo, hi, config)
```

The L_1 norm seems to prevent premature termination (L_2 sometimes leads to that).

# BlackBoxOptim

```
using BlackBoxOptim
res = bboptimize(g1 ā f; SearchRange = tuple.(lo, hi), MaxFuncEvals = 200,
method = :dnes)
res = bboptimize(g2 ā f; SearchRange = tuple.(lo, hi), MaxFuncEvals = 200,
method = :dxnes)
```

Early termination with not very good values. Some papers suggest that DE and DXNES algorithms should work for these problems, am I tuning them wrong?

# CMA-ES

```
using Evolutionary
cmaes(g1 ā f, 6; Ī¼ = 2, Ī» = 4, tol = 1e-5)
```

terminates somewhat early and does not find a good optimum.

# Searching a Sobol grid

Using a (yet unregistered) package I wrote for this:

```
using SobolSearches
s = sobol_search(g1 ā f, lo, hi, 10, 1000)
s[1]
```

This is fairly robust but I am worried it will not scale with a larger dimension.