# BlackBoxOptim - beginner's questions

**URL:** <https://discourse.julialang.org/t/blackboxoptim-beginners-questions/37013>\
**Category:** Optimization (Mathematical)\
**Tags:** question, optimization\
**Created:** [April 4, 2020, 11:56am UTC](https://discourse.julialang.org/t/blackboxoptim-beginners-questions/37013 "2020-04-04T11:56:29Z")\
**Posts on this page:** 17\
**Page:** 1

<div class="post-metadata">

**Author:** ![roi.holtzman](https://avatars.discourse-cdn.com/v4/letter/r/f05b48/32.png) [@roi.holtzman](https://discourse.julialang.org/u/roi.holtzman)\
**Post date:** [April 4, 2020, 11:56am UTC](https://discourse.julialang.org/t/blackboxoptim-beginners-questions/37013/1 "2020-04-04T11:56:29Z")

</div>

I am new to solving optimization problems. So please excuse any ignorance in my questions.

I am using [BlackBoxOptim.jl](https://github.com/robertfeldt/BlackBoxOptim.jl), and I have a few questions:

1. Initial guess and search range.  
I see that there is an optional argument of `SearchRange`. What happens when no range is specified? What is the initial guess? Is it random or deterministic? Is there a way to control the initial guess?

2. Stopping condition.  
In case I do not know the minimum of the function, and therefore cannot use the `TargetFitness` parameter, what is the stopping condition of the optimizer? I guess that if it gets to more than `MaxSteps` iterations, it stops. But is there a way to provide other conditions? Like the wanted tolerance of the fitness function?

3. Keeping track of the results.  
Is there a way to log the attempted values by the optimizer? I am interested to see which regions in the parameter space were explored, and what was the value of the fitness function.

4. Choosing an optimizer.  
I do not know how to choose an optimizer. I read in the documentation the [guide](https://github.com/robertfeldt/BlackBoxOptim.jl#guide-to-selecting-an-optimizer) so I am using the recommended `adaptive_de_rand_1_bin_radiuslimited` and `de_rand_1_bin DE`. To that I compare the results of the `random_search`. Is there some more advice on how to choose / compare between the optimizers? (My problem has dimensions of O(100)).

---

<div class="post-metadata">

**Author:** ![peterj](https://avatars.discourse-cdn.com/v4/letter/p/22d042/32.png) [@peterj](https://discourse.julialang.org/u/peterj)\
**Post date:** [April 6, 2020, 9:14pm UTC](https://discourse.julialang.org/t/blackboxoptim-beginners-questions/37013/2 "2020-04-06T21:14:36Z")

</div>

I’m in the same boat and have a lot of similar questions, but I can also help with some answers maybe.

1. It seems like the initial guess is randomly drawn from the initial range, which is (-1,1) if no range is provided.

2. I just modified my objective function to print its arguments, which seems to work fine.

---

<div class="post-metadata">

**Author:** ![roi.holtzman](https://avatars.discourse-cdn.com/v4/letter/r/f05b48/32.png) [@roi.holtzman](https://discourse.julialang.org/u/roi.holtzman)\
**Post date:** [April 13, 2020, 4:43pm UTC](https://discourse.julialang.org/t/blackboxoptim-beginners-questions/37013/3 "2020-04-13T16:43:59Z")

</div>

Thanks for the input!

---

<div class="post-metadata">

**Author:** ![robertfeldt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/robertfeldt/32/239_2.png) [@robertfeldt](https://discourse.julialang.org/u/robertfeldt)\
**Post date:** [April 19, 2020, 6:52am UTC](https://discourse.julialang.org/t/blackboxoptim-beginners-questions/37013/4 "2020-04-19T06:52:47Z")

</div>

Sorry for missing these questions earlier. Anyway:

1. Default SearchRange is (-1.0, 1.0). Default sampling is [latin hypersquare sampling](https://en.wikipedia.org/wiki/Latin_hypercube_sampling) with the goal being a better “spread” over the search space. There is currently no easy way to provide an initial guess but this is one of the most requested features so we are currently working on that.

2. MaxSteps and MaxTime are the most commonly used stopping conditions. But you can also use a callback function and explicitly stop based on your own logic. There is an example of this in the test suite, see bottom of: [fixed shutdown methods and added test shutting down optimization from… · robertfeldt/BlackBoxOptim.jl@5ed8cd9 · GitHub](https://github.com/robertfeldt/BlackBoxOptim.jl/commit/5ed8cd9dc3f71c984440874d9de773b6fc28a3f7#diff-c97f30bd0122b629381a26ebb5cbec0e)

3. There is logging and tracing but they are not very flexible. If you want to log everything you can either do this in the fitness function or use a callback function, see for example: [BlackBoxOptim.jl/save\_fitness\_progress\_via\_callback\_function.jl at master · robertfeldt/BlackBoxOptim.jl · GitHub](https://github.com/robertfeldt/BlackBoxOptim.jl/blob/master/examples/save_fitness_progress_via_callback_function.jl)

4. The default choice (`adaptive_de_rand_1_bin_radiuslimited`) is very robust and in particular when your problem is large. You can also try `dxnes` (can be more effective if smaller problem) or `generating_set_search` (can be faster for “simpler” problems). The other ones rarely have an edge on any of these three in my experience but YMMV. A new CMA-ES is coming soon which I have some high hopes for.

Hope this helps!

---

<div class="post-metadata">

**Author:** ![compleat](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/compleat/32/8958_2.png) [@compleat](https://discourse.julialang.org/u/compleat)\
**Post date:** [June 19, 2020, 1:58pm UTC](https://discourse.julialang.org/t/blackboxoptim-beginners-questions/37013/5 "2020-06-19T13:58:48Z")

</div>

Hi I have another beginner’s question.

I have one version where using Tuples for the SearchRange  
(like in the introductory Rosenbrock example) works, and another version of BlackBoxOptim where it rejects the use of Tuples for SearchRange.

I could not find any documentation for clarification of how to specify ranges.  
[update: I figured out that the ranges cannot be Integers - you have to add the decimal points! Tuples are OK now]

---

<div class="post-metadata">

**Author:** ![bienpierre](https://avatars.discourse-cdn.com/v4/letter/b/8c91f0/32.png) [@bienpierre](https://discourse.julialang.org/u/bienpierre)\
**Post date:** [July 9, 2020, 1:43pm UTC](https://discourse.julialang.org/t/blackboxoptim-beginners-questions/37013/6 "2020-07-09T13:43:51Z")

</div>

I am also new with BlackBoxOptim.jl

Where can I find the references about optimizers?

Warmest Regards

---

<div class="post-metadata">

**Author:** ![robertfeldt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/robertfeldt/32/239_2.png) [@robertfeldt](https://discourse.julialang.org/u/robertfeldt)\
**Post date:** [August 17, 2020, 1:58pm UTC](https://discourse.julialang.org/t/blackboxoptim-beginners-questions/37013/7 "2020-08-17T13:58:37Z")

</div>

Sorry for late reply; vacation here…

Anyway, documentation is lacking but if you want to see which methods are implemented you can check:

[https://github.com/robertfeldt/BlackBoxOptim.jl/blob/master/src/optimization\_methods.jl](https://github.com/robertfeldt/BlackBoxOptim.jl/blob/master/src/optimization_methods.jl)

Note though that the DE variants for single-objective and BORG for multi-objective have seen the most use and testing.

Hope this helps.

---

<div class="post-metadata">

**Author:** ![mengxiaoliu](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mengxiaoliu/32/23601_2.png) [@mengxiaoliu](https://discourse.julialang.org/u/mengxiaoliu)\
**Post date:** [July 13, 2021, 2:11pm UTC](https://discourse.julialang.org/t/blackboxoptim-beginners-questions/37013/8 "2021-07-13T14:11:29Z")

</div>

I have another question for @robertfeldt:  
Is it possible to add an inequality constraint for two parameters, e.g., x[1]\>x[2]?  
Thanks!

---

<div class="post-metadata">

**Author:** ![robertfeldt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/robertfeldt/32/239_2.png) [@robertfeldt](https://discourse.julialang.org/u/robertfeldt)\
**Post date:** [July 13, 2021, 7:09pm UTC](https://discourse.julialang.org/t/blackboxoptim-beginners-questions/37013/9 "2021-07-13T19:09:32Z")

</div>

These types of optimizers don’t support that out of the box so the way to do it is typically to add some large penalty for violating the constraint or use a multi-objective optimizer and have the second fitness be the penalty for violating the constraint. Some code to give you an idea (this is for the first option, can be adapted for the 2nd option):

```julia
const BasePenalty = 10000 # Select some large number that will dominate the "normal" fitness.

function penalty_constraint_x1_gt_x2(x)
    if x[1] < x[2]
        return BasePenalty + (x[2] - x[1]) # Smaller penalty as we get closer to non-violation of constraint
    else
        return 0.0
    end
end

my_new_fitness(x) = my_normal_fitness(x) + penalty_constraint_x1_gt_x2(x)

res = bboptimize(my_new_fitness; ...)

```

---

<div class="post-metadata">

**Author:** ![mengxiaoliu](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mengxiaoliu/32/23601_2.png) [@mengxiaoliu](https://discourse.julialang.org/u/mengxiaoliu)\
**Post date:** [July 13, 2021, 7:28pm UTC](https://discourse.julialang.org/t/blackboxoptim-beginners-questions/37013/10 "2021-07-13T19:28:21Z")

</div>

Got it. Thanks!

---

<div class="post-metadata">

**Author:** ![mengxiaoliu](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mengxiaoliu/32/23601_2.png) [@mengxiaoliu](https://discourse.julialang.org/u/mengxiaoliu)\
**Post date:** [August 4, 2021, 3:09pm UTC](https://discourse.julialang.org/t/blackboxoptim-beginners-questions/37013/11 "2021-08-04T15:09:08Z")

</div>

I log the optimization progress (including fitness value and candidates) in my fitness function, but I am confused about what value is actually recorded. For example, I choose my population size to be 1000 and perform 10000 evaluations. I expect the log file to contain 1000\*10000=10e07 records because each evaluation should input 1000 candidates to the fitness function. However, the log file contains around 10000 fitness values, and the best fitness value in the log file seems to perform worse than the best fitness value in the optimization output.  
Could you explain this to me, please? Thank you!

---

<div class="post-metadata">

**Author:** ![mengxiaoliu](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mengxiaoliu/32/23601_2.png) [@mengxiaoliu](https://discourse.julialang.org/u/mengxiaoliu)\
**Post date:** [August 4, 2021, 6:44pm UTC](https://discourse.julialang.org/t/blackboxoptim-beginners-questions/37013/12 "2021-08-04T18:44:30Z")

</div>

I think I know why now. The number of evaluations applies to each candidate in each population. So for a full generation to evolve, I should select the number of evaluations as an integer multiplying the population size. For example, if I selected the population size to be 50 and I want the population to evolve 1000 times, I should set my evaluations to be 50\*1000 = 50,000.

---

<div class="post-metadata">

**Author:** ![mehdi\_lotfi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mehdi_lotfi/32/37774_2.png) [@mehdi\_lotfi](https://discourse.julialang.org/u/mehdi_lotfi)\
**Post date:** [October 26, 2022, 10:14am UTC](https://discourse.julialang.org/t/blackboxoptim-beginners-questions/37013/13 "2022-10-26T10:14:16Z")

</div>

This post was temporarily hidden by the community for possibly being off-topic, unfocused, inappropriate, or spammy.

---

<div class="post-metadata">

**Author:** ![robertfeldt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/robertfeldt/32/239_2.png) [@robertfeldt](https://discourse.julialang.org/u/robertfeldt)\
**Post date:** [October 26, 2022, 10:29am UTC](https://discourse.julialang.org/t/blackboxoptim-beginners-questions/37013/14 "2022-10-26T10:29:49Z")

</div>

Hi,

Not sure what you want to achieve since your example code is a bit confusing.

Currently you state that the number of dimensions is the length of p\_init. But if p\_init is a vector of the initial guesses the dimension is the length of one of those vectors. So it seems you want

```julia
NumDimensions = length(first(p_init))

```

Please see the example code in the README of the package. It shows an example when we give a single initial guess (a vector of two floats since our dimension is 2) or when we give a vector of two such initial guesses.

Hope this helps.

---

<div class="post-metadata">

**Author:** ![mehdi\_lotfi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mehdi_lotfi/32/37774_2.png) [@mehdi\_lotfi](https://discourse.julialang.org/u/mehdi_lotfi)\
**Post date:** [October 26, 2022, 10:43am UTC](https://discourse.julialang.org/t/blackboxoptim-beginners-questions/37013/15 "2022-10-26T10:43:10Z")

</div>

Thank you for your response. My main problem is how I can define bboptimize for the vectorized problems. For example, suppose we want to estimate parameters with the type of Vector{Vector{Float64}}, so how can we define bboptimize for this?  
Note: The initial values for parameters are like this  
p\_init =  
[  
[0.2, 0.2, 0.2, 0.2, 0.2, 0.2],  
[0.4, 0.4, 0.4, 0.4, 0.4, 0.4],  
[0.3, 0.3, 0.3, 0.3, 0.3, 0.3]  
]

---

<div class="post-metadata">

**Author:** ![robertfeldt](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/robertfeldt/32/239_2.png) [@robertfeldt](https://discourse.julialang.org/u/robertfeldt)\
**Post date:** [October 26, 2022, 11:20am UTC](https://discourse.julialang.org/t/blackboxoptim-beginners-questions/37013/16 "2022-10-26T11:20:03Z")

</div>

BlackBoxOptim always optimises a flat vector of floats. But you can just reshape it before you apply your fitness function. So, basically use vcat to turn a vector of vector into a straight vector and then reshape (or just view) before using your fitness function. Code to inspire you:

```julia
julia> p_init = [
       [0.2, 0.2, 0.2, 0.2, 0.2, 0.2],
       [0.4, 0.4, 0.4, 0.4, 0.4, 0.4],
       [0.3, 0.3, 0.3, 0.3, 0.3, 0.3]
       ]
3-element Vector{Vector{Float64}}:
 [0.2, 0.2, 0.2, 0.2, 0.2, 0.2]
 [0.4, 0.4, 0.4, 0.4, 0.4, 0.4]
 [0.3, 0.3, 0.3, 0.3, 0.3, 0.3]

julia> p_init_as_vector = vcat(p_init...)
18-element Vector{Float64}:
 0.2
 0.2
...
 0.3
 0.3

julia> function asvectors(x::Vector{Float64}, numvecs, lenvec)
           map(s -> view(x, s:(s+lenvec-1)), 1:lenvec:(numvecs*lenvec))
           end
asvectors (generic function with 1 method)

julia> asvectors(p_init_as_vector, 3, 6)
3-element Vector{SubArray{Float64, 1, Vector{Float64}, Tuple{UnitRange{Int64}}, true}}:
 [0.2, 0.2, 0.2, 0.2, 0.2, 0.2]
 [0.4, 0.4, 0.4, 0.4, 0.4, 0.4]
 [0.3, 0.3, 0.3, 0.3, 0.3, 0.3]

```

---

<div class="post-metadata">

**Author:** ![jgr](https://avatars.discourse-cdn.com/v4/letter/j/cdc98d/32.png) [@jgr](https://discourse.julialang.org/u/jgr)\
**Post date:** [January 12, 2025, 9:07pm UTC](https://discourse.julialang.org/t/blackboxoptim-beginners-questions/37013/17 "2025-01-12T21:07:18Z")

</div>

Hi, can I ask a question about BlackBoxOptim? Every time I use it to solve the solution, I always get different results. I think that is because the stochastic property of the DE algorithms. I was wondering if there is any way we could set the seeds? I tried directly adding `rng = MersenneTwister(123)`, but it does not seem to work. Thank you!