# Providing More Initial Parameter Values than there are Parameters in the Function Being Optimized

**URL:** <https://discourse.julialang.org/t/providing-more-initial-parameter-values-than-there-are-parameters-in-the-function-being-optimized/45585>\
**Category:** Optimization (Mathematical)\
**Tags:** question, diffeq\
**Created:** [August 26, 2020, 5:53pm UTC](https://discourse.julialang.org/t/providing-more-initial-parameter-values-than-there-are-parameters-in-the-function-being-optimized/45585 "2020-08-26T17:53:06Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![donkeysaddle](https://avatars.discourse-cdn.com/v4/letter/d/3e96dc/32.png) [@donkeysaddle](https://discourse.julialang.org/u/donkeysaddle)\
**Post date:** [August 26, 2020, 5:53pm UTC](https://discourse.julialang.org/t/providing-more-initial-parameter-values-than-there-are-parameters-in-the-function-being-optimized/45585/1 "2020-08-26T17:53:06Z")

</div>

Why does the “extra” parameter value affect the optimization? Apologies if this is in the Optim documentation - I didn’t see it.

For example:

```julia
using Optim
f(x) = (1.0 - x[1])^2 + 100.0 * (x[2] - x[1]^2)^2

x0 = [0.0, 0.0, 2.0]
optimize(f, x0)

```

gives a different result than

```julia
f(x) = (1.0 - x[1])^2 + 100.0 * (x[2] - x[1]^2)^2

x0 = [0.0, 0.0]
optimize(f, x0)

```

What is the “2.0” in the initial parameter value array doing?

---

<div class="post-metadata">

**Author:** ![juliohm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juliohm/32/215266_2.png) [@juliohm](https://discourse.julialang.org/u/juliohm)\
**Post date:** [August 26, 2020, 6:27pm UTC](https://discourse.julialang.org/t/providing-more-initial-parameter-values-than-there-are-parameters-in-the-function-being-optimized/45585/2 "2020-08-26T18:27:03Z")

</div>

I don’t know what it is doing, but probably the internals of Optim.jl are general enough so that the code doesn’t care about the exact length of the vector on updates. I would simply avoid passing the incorrect vector to the `optimize` function if it is a 2D objective.

---

<div class="post-metadata">

**Author:** ![donkeysaddle](https://avatars.discourse-cdn.com/v4/letter/d/3e96dc/32.png) [@donkeysaddle](https://discourse.julialang.org/u/donkeysaddle)\
**Post date:** [August 26, 2020, 6:33pm UTC](https://discourse.julialang.org/t/providing-more-initial-parameter-values-than-there-are-parameters-in-the-function-being-optimized/45585/3 "2020-08-26T18:33:15Z")

</div>

Could it be changing f(x) to this:

```julia
f(x) = (1.0 - x[1])^2 + 100.0 * (x[2] - x[1]^2)^2 + 0*x[3]

```

Would be nice to know - I made this mistake with a more complicated problem and the results I got were actually improved…

---

<div class="post-metadata">

**Author:** ![juliohm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juliohm/32/215266_2.png) [@juliohm](https://discourse.julialang.org/u/juliohm)\
**Post date:** [August 26, 2020, 6:36pm UTC](https://discourse.julialang.org/t/providing-more-initial-parameter-values-than-there-are-parameters-in-the-function-being-optimized/45585/4 "2020-08-26T18:36:16Z")

</div>

We can play the guessing game, but if you are really interested in this corner case, Optim.jl is open source! Type `@edit optimize(f, x0)` and read the code 😉

The issue is probably not in the objective function, but in the vector updates. You can write `x -= M*x` for example without knowing the exact length of `x`.

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [August 27, 2020, 8:47am UTC](https://discourse.julialang.org/t/providing-more-initial-parameter-values-than-there-are-parameters-in-the-function-being-optimized/45585/5 "2020-08-27T08:47:36Z")

</div>

> [@donkeysaddle](#):
>
> gives a different result than

I can’t replicate this. I get the same minimizer (within numerical error) for the first two coordinates. Of course the third coordinate is random, but that should not matter.

If you are asking whey the results are not _exactly_ equal: they are two different problems (in \mathbb{R}^2 and \mathbb{R}^3) and different steps are taken.

---

<div class="post-metadata">

**Author:** ![yha](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yha/32/3502_2.png) [@yha](https://discourse.julialang.org/u/yha)\
**Post date:** [August 27, 2020, 11:21am UTC](https://discourse.julialang.org/t/providing-more-initial-parameter-values-than-there-are-parameters-in-the-function-being-optimized/45585/6 "2020-08-27T11:21:46Z")

</div>

> [@donkeysaddle](#):
>
> Could it be changing f(x) to this:

Why would you think it changes `f`? The function you originally defined is applicable to vectors of length 3 (or any length ≥2) as is:

```julia
f(x) = (1.0 - x[1])^2 + 100.0 * (x[2] - x[1]^2)^2
f([1,2,3]) ## == 100.0

```

---

<div class="post-metadata">

**Author:** ![pkofod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pkofod/32/2179_2.png) [@pkofod](https://discourse.julialang.org/u/pkofod)\
**Post date:** [August 27, 2020, 12:45pm UTC](https://discourse.julialang.org/t/providing-more-initial-parameter-values-than-there-are-parameters-in-the-function-being-optimized/45585/7 "2020-08-27T12:45:33Z")

</div>

Well, I don’t see why it would affect the result. Are you sure you don’t have a sum of x or norm or something like that that would be affected by the third element?

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [August 27, 2020, 1:31pm UTC](https://discourse.julialang.org/t/providing-more-initial-parameter-values-than-there-are-parameters-in-the-function-being-optimized/45585/8 "2020-08-27T13:31:21Z")

</div>

> [@pkofod](#):
>
> Well, I don’t see why it would affect the result.

I don’t think it does (in the \approx sense). Did you run the example?

---

<div class="post-metadata">

**Author:** ![pkofod](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/pkofod/32/2179_2.png) [@pkofod](https://discourse.julialang.org/u/pkofod)\
**Post date:** [August 27, 2020, 1:46pm UTC](https://discourse.julialang.org/t/providing-more-initial-parameter-values-than-there-are-parameters-in-the-function-being-optimized/45585/9 "2020-08-27T13:46:10Z")

</div>

Ah, so actually it does in this case because you’re using Nelder-Mead and the initial simplex is affected by the initial `x`. The centroid element for `x[1]` and `x[2]` changes which changes the progression of the algorithm. Edit: but the solution is obviously approximately the same (one has element just above one and the other just below, and the minimizer is [1,1])

---

<div class="post-metadata">

**Author:** ![donkeysaddle](https://avatars.discourse-cdn.com/v4/letter/d/3e96dc/32.png) [@donkeysaddle](https://discourse.julialang.org/u/donkeysaddle)\
**Post date:** [September 3, 2020, 5:51pm UTC](https://discourse.julialang.org/t/providing-more-initial-parameter-values-than-there-are-parameters-in-the-function-being-optimized/45585/10 "2020-09-03T17:51:45Z")

</div>

Thanks for all the responses. This makes sense, much more plausible than my hypothesis.

Thanks again,

DS