# Optimization using SAMIN()

**URL:** <https://discourse.julialang.org/t/optimization-using-samin/25351>\
**Category:** Optimization (Mathematical)\
**Created:** [June 17, 2019, 1:06am UTC](https://discourse.julialang.org/t/optimization-using-samin/25351 "2019-06-17T01:06:54Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![Farnazmdi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/farnazmdi/32/24072_2.png) [@Farnazmdi](https://discourse.julialang.org/u/Farnazmdi)\
**Post date:** [June 17, 2019, 1:06am UTC](https://discourse.julialang.org/t/optimization-using-samin/25351/1 "2019-06-17T01:06:54Z")

</div>

Hi,

I am trying to use SAMIN() for constrained optimization for a DDE parameter estimation, I have seen the example in [[Optim.jl](http://julianlsolvers.github.io/Optim.jl/latest/#algo/samin/)]  
and I am following the same syntax, but get the error of Dimension mismatch.

`resid` is the residuals function, the difference of the model and the data, and `p` is the initial guess.  
Here are my code and error. I appreciate if you could help me with this error.

 ![image](https://global.discourse-cdn.com/julialang/original/3X/2/9/290a77a7d0a45ba3aae40634a16d4f47922ad04f.png)

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [June 17, 2019, 2:31am UTC](https://discourse.julialang.org/t/optimization-using-samin/25351/2 "2019-06-17T02:31:49Z")

</div>

`res[1,:] = sol(t, idxs=1) - G1_new` etc. should be sufficient. But the optimizer likely wants a scalar output, so you need to output something like `norm(res)`.

---

<div class="post-metadata">

**Author:** ![Farnazmdi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/farnazmdi/32/24072_2.png) [@Farnazmdi](https://discourse.julialang.org/u/Farnazmdi)\
**Post date:** [June 17, 2019, 3:03am UTC](https://discourse.julialang.org/t/optimization-using-samin/25351/3 "2019-06-17T03:03:49Z")

</div>

Thank you so much, Chris. It worked, although, not converging.

---

<div class="post-metadata">

**Author:** ![pkofod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pkofod/32/2179_2.png) [@pkofod](https://discourse.julialang.org/u/pkofod)\
**Post date:** [June 17, 2019, 7:51am UTC](https://discourse.julialang.org/t/optimization-using-samin/25351/4 "2019-06-17T07:51:02Z")

</div>

How many “iterations” and what parameters do you use?

---

<div class="post-metadata">

**Author:** ![Farnazmdi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/farnazmdi/32/24072_2.png) [@Farnazmdi](https://discourse.julialang.org/u/Farnazmdi)\
**Post date:** [June 17, 2019, 4:43pm UTC](https://discourse.julialang.org/t/optimization-using-samin/25351/5 "2019-06-17T16:43:47Z")

</div>

I tried 10^4 iterations, it threw a warning that it is not stable (?) I have 9 parameters, basically, it is a two-equations delay differential equation model, I am also trying to estimate the initial values and the coefficient of h(p, t), as well as the coefficient of other terms.  
I tried to use SAMIN() for a simpler one, which only had two parameters, but it didn’t converge either.

I am just passing the norm of the difference between data and the mode, is it correct to do that? I tried the sum of squared errors, but it worked worse.  
This is my model:

```julia
function G1_G2(du, u, h, p, t)
    du[1] = -p[1]*(h(p, t-p[5])[1]) + 2*p[2]*(h(p, t-p[6])[2]) - p[3]*u[1]
    du[2] = p[1]*(h(p, t-p[5])[1]) - p[2]*(h(p, t-p[6])[2]) - p[4]*u[2] 
end

```

I have data for both u[1] and u[2], I was giving both to the optimizer with `Dogleg()` but here since it has to be a scalar, I am just passing norm of one of them.

 ![image](https://global.discourse-cdn.com/julialang/original/3X/1/1/1101342822faab8da414afbe0d191008ec80795a.png)  
Here is the summary:

---

<div class="post-metadata">

**Author:** ![mcreel](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mcreel/32/30088_2.png) [@mcreel](https://discourse.julialang.org/u/mcreel)\
**Post date:** [June 17, 2019, 5:30pm UTC](https://discourse.julialang.org/t/optimization-using-samin/25351/6 "2019-06-17T17:30:19Z")

</div>

In that output, it looks like the iter limit is 10^3. Try setting it to 10^8, there’s no reason to set it low, unless function evaluation is too costly. If you find it’s taking too long, try setting rt=0.5. Tuning rt is a bit of an art, the safe procedure is to leave it not too far below 1, but too close leads to very slow convergence.

---

<div class="post-metadata">

**Author:** ![Farnazmdi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/farnazmdi/32/24072_2.png) [@Farnazmdi](https://discourse.julialang.org/u/Farnazmdi)\
**Post date:** [June 17, 2019, 5:33pm UTC](https://discourse.julialang.org/t/optimization-using-samin/25351/7 "2019-06-17T17:33:11Z")

</div>

I tried to set the iterations to a 10^6 it was taking forever, I thought something might be wrong.  
I will do it. Thank you very much.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [June 17, 2019, 7:20pm UTC](https://discourse.julialang.org/t/optimization-using-samin/25351/8 "2019-06-17T19:20:42Z")

</div>

> [@mcreel](#):
>
> unless function evaluation is too costly

it’s solving a DDE each time. That can get costly.

---

<div class="post-metadata">

**Author:** ![mcreel](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mcreel/32/30088_2.png) [@mcreel](https://discourse.julialang.org/u/mcreel)\
**Post date:** [June 18, 2019, 4:57am UTC](https://discourse.julialang.org/t/optimization-using-samin/25351/9 "2019-06-18T04:57:46Z")

</div>

Well, setting a low iteration limit just won’t work with SA, unless rt is also adjusted downward so that the algorithm focuses in quickly. And, doing this, you’re unlikely to find the global min unless the objective function is fairly regular. If you want the right answer and your problem is costly, you’re going to have to be willing to wait a bit.

If you do set a low iteration limit, then you need to set rt low enough so that it converges within the limit. Then, if you run it again a few times and get the same result, you have some evidence that the solution is a global min.

---

<div class="post-metadata">

**Author:** ![jonathanBieler](https://avatars.discourse-cdn.com/v4/letter/j/82dd89/32.png) [@jonathanBieler](https://discourse.julialang.org/u/jonathanBieler)\
**Post date:** [June 18, 2019, 8:33am UTC](https://discourse.julialang.org/t/optimization-using-samin/25351/10 "2019-06-18T08:33:27Z")

</div>

I usually find it useful to plot the optimisation trace to understand what’s going on, but SAMIN seems to store nothing, is that an oversight ?

```julia
mfit = optimize(
    x->sum(abs2,x),fill(-5.,3),fill(5.,3),randn(3), Optim.SAMIN(),
    Optim.Options(f_calls_limit=10_000,store_trace=true,iterations=10_000,extended_trace=true),

julia> mfit.trace
0-element Array{OptimizationState{Float64,SAMIN{Float64}},1}
)

```

---

<div class="post-metadata">

**Author:** ![pkofod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pkofod/32/2179_2.png) [@pkofod](https://discourse.julialang.org/u/pkofod)\
**Post date:** [June 18, 2019, 10:00am UTC](https://discourse.julialang.org/t/optimization-using-samin/25351/11 "2019-06-18T10:00:31Z")

</div>

I’m 23 days ahead of you [https://github.com/JuliaNLSolvers/Optim.jl/commit/d9fbf09db2fc7f8ba0ff7e47aeaa737f9ec3eeea](https://github.com/JuliaNLSolvers/Optim.jl/commit/d9fbf09db2fc7f8ba0ff7e47aeaa737f9ec3eeea) but way behind on tagging 🙂  
edit: let me figure out registrator  
edit2: i guess i need a Project file. I’ll try to fix that tonight