# \[ANN-RFC\] NaturalES.jl - Natural Gradient Optimization
**URL:** https://discourse.julialang.org/t/ann-rfc-naturales-jl-natural-gradient-optimization/38043
**Category:** Package Announcements
**Created:** [April 22, 2020, 8:45pm UTC](https://discourse.julialang.org/t/ann-rfc-naturales-jl-natural-gradient-optimization/38043 "2020-04-22T20:45:49Z")
**Posts on this page:** 7
**Page:** 1
### Author:  [@francesco.alemanno](https://discourse.julialang.org/u/francesco.alemanno)
#### Post date: [April 22, 2020, 8:45pm UTC](https://discourse.julialang.org/t/ann-rfc-naturales-jl-natural-gradient-optimization/38043/1 "2020-04-22T20:45:49Z")
> **[GitHub - francescoalemanno/NaturalES.jl: Simple julia Natural Evolution Strategies...](https://github.com/francescoalemanno/NaturalES.jl)**
>
> Simple julia Natural Evolution Strategies implementation
# NaturalES.jl
This package implements the optimization methods described in
[_Wierstra, et al “Natural Evolution Strategies”, JMLR (2014)_](http://www.jmlr.org/papers/volume15/wierstra14a/wierstra14a.pdf).
this implementation follows the KISS™ principle, it can be used as
# Usage
```julia
function rosenbrock(x::AbstractVector{T}) where T
s=(1.0 - x[1])^2
for i in 1:(length(x)-1)
s+=100.0 * (x[i+1] - x[i]^2)^2
end
return s
end
optimize(rosenbrock,[0.3,0.6],1.0,sNES) # separable natural es.
(sol = [0.9999902815083116, 0.9999805401026993], cost = 9.450201922031972e-11)
optimize(rosenbrock,[0.3,0.6],1.0,xNES) # exponential natural es.
(sol = [0.9999999934969991, 0.9999999871800216], cost = 4.574949214506023e-17)
```
for further info in Julia type `?optimize`.
# Tips:
- Use xNES for hard problems with strongly correlated variables
- Use sNES for high dimensional problems that exhibit many local minima
- Use sNES for problems with mostly separable variables
# Future Plans:
- Implementing other strongly performing variants
- Parallelization
~~if there is any interest in having this package registered, let me know!~~
you can now install this package by typing `]add NaturalES` in the Julia REPL.
---
### Author:  [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)
#### Post date: [April 22, 2020, 10:20pm UTC](https://discourse.julialang.org/t/ann-rfc-naturales-jl-natural-gradient-optimization/38043/2 "2020-04-22T22:20:08Z")
Sweet! How does it compare to [https://github.com/robertfeldt/BlackBoxOptim.jl](https://github.com/robertfeldt/BlackBoxOptim.jl) ?
---
### Author:  [@francesco.alemanno](https://discourse.julialang.org/u/francesco.alemanno)
#### Post date: [April 23, 2020, 3:30pm UTC](https://discourse.julialang.org/t/ann-rfc-naturales-jl-natural-gradient-optimization/38043/3 "2020-04-23T15:30:28Z")
i’d say in some cases it can compare very favorably 😃
```julia
using BenchmarkTools
using BlackBoxOptim
using NaturalES
function rosenbrock(x::AbstractVector{T}) where T
s=(1.0 - x[1])^2
for i in 1:(length(x)-1)
s+=100.0 * (x[i+1] - x[i]^2)^2
end
return s
end
function bb_task()
best_candidate(bboptimize(rosenbrock; SearchRange = (-5.0, 5.0), NumDimensions = 2, Method = :separable_nes,TraceMode=:silent))
end
function nates_task()
optimize(rosenbrock,[0.0,0.0],2.0,sNES).sol
end
@btime bb_task() #19.940 ms (276040 allocations: 14.91 MiB)
@btime nates_task() #652.101 μs (3867 allocations: 61.84 KiB)
print(bb_task()) #[0.9876181286752018, 0.975347963147114]
print(nates_task()) #[0.9999963410120233, 0.9999926866427169]
using LinearAlgebra
print(norm(bb_task().-[1,1])/norm(nates_task().-[1,1])) # ~ 10^3
```
much less time,much less allocations, for a much better solution
in this case my package is ~ 31 times faster and it reaches a solution three order of magnitude more accurate
the argument is still valid also for `xNES`
```julia
function bb_task()
best_candidate(bboptimize(rosenbrock; SearchRange = (-5.0, 5.0), NumDimensions = 2, Method = :xnes,TraceMode=:silent))
end
function nates_task()
optimize(rosenbrock,[0.0,0.0],2.0,xNES).sol
end
@btime bb_task() #126.951 ms (326961 allocations: 26.50 MiB)
@btime nates_task() #175.700 μs (2743 allocations: 233.63 KiB)
```
---
### Author:  [@AndiMD](https://discourse.julialang.org/u/AndiMD)
#### Post date: [April 23, 2020, 6:45pm UTC](https://discourse.julialang.org/t/ann-rfc-naturales-jl-natural-gradient-optimization/38043/4 "2020-04-23T18:45:02Z")
Nice work. Maybe this can be merged into BlackBoxOptim, or vice versa, in the future.
---
### Author:  [@francesco.alemanno](https://discourse.julialang.org/u/francesco.alemanno)
#### Post date: [April 27, 2020, 12:39pm UTC](https://discourse.julialang.org/t/ann-rfc-naturales-jl-natural-gradient-optimization/38043/5 "2020-04-27T12:39:40Z")
[https://github.com/JuliaRegistries/General/pull/13727](https://github.com/JuliaRegistries/General/pull/13727)
~~the package is in the process of being registered.~~
The package is now registered!
thank you everyone for showing some love for this tiny package!
---
### Author:  [@jonathanBieler](https://discourse.julialang.org/u/jonathanBieler)
#### Post date: [May 1, 2020, 1:25pm UTC](https://discourse.julialang.org/t/ann-rfc-naturales-jl-natural-gradient-optimization/38043/6 "2020-05-01T13:25:04Z")
Note that the performance that people care about usually is the probability of finding the global minima in function of the number of function evaluations, not really the time the algorithm needs to run (since the bottleneck if often evaluating the error function).
I have a [small framework](https://github.com/jonathanBieler/BlackBoxOptimizationBenchmarking.jl) to do that kind of benchmarks if you are interested.
---
### Author:  [@francesco.alemanno](https://discourse.julialang.org/u/francesco.alemanno)
#### Post date: [May 1, 2020, 6:34pm UTC](https://discourse.julialang.org/t/ann-rfc-naturales-jl-natural-gradient-optimization/38043/7 "2020-05-01T18:34:09Z")
it would be nice to add my package to those beautiful plots!